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I.  Introduction 


This  is  the  final  report  for  DARPA  grant  DAMD17-94-J-4113  on  the 
introduction  of  olfactory  stimuli  into  virtual  reality  for  medical  training 
applications.  The  project  addresses  the  range  of  issues  that  are  required 
for  such  stimuli  to  be  used  for  this  purpose  including:  odorant  delivery, 
wireless  operation,  odor  representation,  odor  effects  on  performance,  and 
graphic  simulation.  While  the  purpose  of  this  report  is  to  describe  the 
efforts  of  the  past  year  and  the  end  state  of  the  project  as  a  whole,  it  is 
also  desirable  to  describe  enough  of  the  previous  years’  efforts  to  provide 
context  for  what  has  been  done  more  recently. 
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II.  Executive  Summary 


This  report  describes  the  results  of  the  DARPA-sponsored  effort  to 
determine  the  state  of  current  technology  vis  a  vis  the  problem  of 
introducing  olfactory  stimuli  into  virtual  reality  training  simulations. 
Since  no  history  existed  from  which  to  determine  requirements  and  no 
technology  existed  that  was  directly  suitable  for  this  task,  this  project 
was  a  systems  effort  touching  on  a  number  of  technological  problems 
including;  odor  delivery,  wireless  tracking,  odorant  synthesis,  graphics 
simulation,  and  human  performance  studies. 

II. A.  Wireless  Operation 

Given  that  odor  display  is  a  new  capability,  it  was  seen  as  desirable  for 
participants  in  immersive  virtual  reality  system  to  encounter  odors  in  the 
most  natural  way  possible.  This  implied  the  need  for  completely  wireless 
operation  comprising  wireless  image  transmission,  wireless  odor  display, 
and  wireless  determination  of  head  position  and  orientation.  All  of  these 
have  been  achieved  at  different  stages  of  the  project,  although  not  yet  in 
an  integrated  system. 

II. B.  Odor  delivery 

A  major  focus  of  this  effort  was  on  the  development  of  delivery  systems 
which  could  be  used  for  presenting  odors  in  desktop  surgical  simulators, 
HMD-based  immersive  virtual  reality,  and  CAVE  environments.  A  number  of 
delivery  systems  were  developed  and  the  result  of  the  final  iteration  is  a 
single  system  that  can  be  used  in  all  three  virtual  reality  formats. 

The  resulting  delivery  system  is  portable,  wearable,  and  wireless.  It  can 
deliver  up  to  10  odors  singly  or  in  combination.  It  can  also  control  the 
concentration  of  the  displayed  odor  over  at  least  a  100:1  range.  Its 
electronic  response  time  is  1  millisecond  which  is  faster  than  any 
imaginable  head  movement  of  which  a  virtual  reality  participant  might  be 
capable.  This  allows  the  participant  to  move  his  head  towards  or  away 
from  a  virtual  odor  source  and  to  perceive  the  expected  increase  or 
decrease  in  the  strength  of  the  stimulus.  However,  there  are  additional 
sources  of  lag  from  the  wireless  communication  and  the  air  transport 
through  the  tubing  to  the  nose  that  bring  the  total  lag  to  over  50 
milliseconds  which  is  still  fast  enough  for  the  application.  The  system 
operates  over  the  wide  range  of  orientations  that  a  standing  participant 
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might  assume. 

II. C.  Tracking 

For  odors  to  be  effective  in  immersive  virtual  reality  systems,  the 
tracking  of  the  participant'  head  position  and  orientation  should  also  be 
wireless.  Early  in  the  project,  a  40  foot  long  wireless  tracking  path  was 
constructed  based  on  overhead  optical  tracking  of  head-mounted  targets. 
This  system  was  fast  for  position  and  yaw,  but  slower  for  pitch  and  roll, 
although  with  current  fast  framing  cameras,  it  would  be  satisfactory.  A 
second  system  with  spectacular  specifications  was  designed  and  its 
electronics  and  software  algorithms  implemented.  It  created  a  6000  x 
6000  virtual  camera  which  could  provide  a  1000  samples  per  second  with 
sub  millimeter  accuracy  over  a  40  foot  square  area.  Unfortunately,  the 
optics  turned  out  to  be  unreasonably  expensive  to  fabricate.  A  third 
technique  is  being  implemented  which  has  a  CAVE-sized  working  volume 
but  which  does  provide  a  high  sampling  rate,  stable  values,  and  wireless 
operation.  It  works  in  benign  orientations  and  is  being  extended  to  make  it 
more  robust. 

li.D.  Odor  Synthesis 

Our  initial  partners  assured  us  that  getting  the  odors  needed  would  not  be 
difficult.  This  turned  out  to  be  incorrect.  While  the  light  molecules  found 
in  flowers  and  fruits  are  well  understood  and  vegetation  should  be 
tractable  with  existing  techniques,  the  heavier  chemicals  in  tissues  and 
bodily  fluids  have  not  received  much  attention  and  are  not  of  interest  to 
the  fragrance  industry.  The  very  mention  of  fuels,  combustion  products, 
and  explosives  strikes  fear  in  the  hearts  of  those  with  the  skills  to 
synthesize  them.  Moreover,  many  of  the  synthesized  odors  that  we  were 
able  to  get  hold  of  or  to  get  formulated  for  us  were  not  very  convincing. 
There  is  a  need  for  a  breakthrough  in  this  area. 

lI.E.  Odor  Studies 

Two  major  studies  were  conducted  to  determine  the  effects  of  odors  on 
skills  needed  to  perform  surgery.  The  first  showed  that  odors  could 
improve  spatial  reasoning  to  a  degree  that  was  of  both  statistical  and 
practical  significance.  The  second  study  showed  no  significant 
improvement  in  the  performance  of  manual  dexterity  tasks  from  the 
presence  of  any  of  the  odors  considered. 
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III.  Wireless  Odor  Display  System 

The  final  odor  display  system  is  the  deliverable  for  this  project.  Its 
purpose  is  to  present  odors  to  a  participant  in  an  immersive  HMD  system 
at  appropriate  moments  in  virtual  reality  training  experiences. 

While  it  was  not  appreciated  at  the  beginning  of  the  project,  this  device  is 
quite  unusual  in  that  it  is  a  portable  fluid  and  vapor  handling  system  that 
works  with  a  wide  variety  of  corrosive  chemicals  and  must  operate  in  a 
wide  range  of  orientations.  Even  defining  the  requirements  was  not 
straightforward,  for  many  of  the  needs  could  only  be  ascertained  by  having 
a  delivery  device  with  which  to  measure  them.  For  instance,  the  amount  of 
odorant  that  needed  to  be  added  to  an  air  stream  breathed  through  a  mask 
or  whether  pulse  width  modulation  of  odor  release  could  be  used  to 
control  concentration  were  not  to  be  found  in  any  publication.  Indeed,  the 
level  of  instrumentation  used  in  past  olfactory  research  makes  some  of  it 
suspect.  Multiple  odor  delivery  and  head-tracked  delivery  had  been  only 
crudely  attempted  if  at  all. 

III. A.  Presentation  Formats 

Three  presentation  formats  were  considered  for  odor  delivery  during  the 
project: 

•  an  enclosed  training  booth  for  surgical  training 

•  a  portable,  wireless  unit  that  could  be  worn  by  users  as  they  participate 
in  training  in  a  simulator  for  dismounted  personnel  and  which  could 
deliver  odors  contingent  on  their  body  movements  and  their  location  and 
events  in  the  virtual  environment. 

•  a  CAVE  environment  in  which  the  participant  is  surrounded  by  projection 
screens  and  his  head  and  hand  movements  are  tracked  as  he  moves  around 
a  10’x10’  space. 

Since  the  training  booth  is  quite  small  and  the  odor  output  can  be  placed 
close  to  the  user’s  face  like  a  microphone,  it  was  found  that  the  portable 
delivery  system  designed  for  immersive  environments  could  also  be  used 
in  this  space  provided  that  a  ventilation  panel  was  placed  behind  the 
participant’s  head  to  evacuate  the  odor  immediately  after  it  was 
introduced. 

On  the  other  hand,  the  CAVE  contains  an  enormous  amount  of  air  and 
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demands  a  much  different  approach  if  the  entire  volume  is  to  be  odorized. 
Several  such  room  odorizing  systems  were  tested  in  a  CAVE-sized  room.  It 
was  found  that  not  only  was  it  difficult  to  evacuate  the  odors  once  they 
were  introduced,  it  was  also  difficult  to  introduce  the  odors  so  that  they 
would  be  simultaneously  available  everywhere  in  the  CAVE.  Indeed,  it 
would  take  a  minute  for  a  new  odor  to  circulate  around  the  space.  Thus,  by 
the  time  a  person  standing  in  the  corners  of  the  room  detected  a  new  odor, 
people  standing  in  the  center  would  already  have  habituated  to  it. 

It  is  possible  to  improve  the  performance  somewhat  by  introducing  the 
odors  uniformly  (no  small  trick)  through  a  porous  floor  and  then  to 
evacuate  them  immediately  through  a  porous  ceiling.  The  problem  is  that 
CAVE  environments  are  open  at  the  top  to  allow  an  image  to  be  projected 
onto  the  floor.  They  are  also  open  at  one  end  so  participants  can  enter  and 
exit.  Laminar  flow  could  only  be  maintained  by  either  giving  up  the 
imagery  on  the  floor  or  by  rear  projecting  it  from  below.  The  result  is  a 
complete  redesign  of  the  CAVE  which  is  not  likely  to  be  welcome  to 
owners  of  existing  CAVEs  or  even  to  appeal  to  those  building  new  ones. 

From  1984-88,  the  PI  implemented  a  gesture-controlled  highly  interactive 
virtual  wind  tunnel  for  Pratt  &  Whitney.[Krueger][Foley]  (This  project  was 
the  direct  inspiration  for  NASA’s  virtual  wind  tunnel.  Both  were 
demonstrated  in  Tomorrow’s  Realities  Gallery  at  SIGGRAPH  91.)  One  small 
experiment  with  this  system  was  a  study  of  rectangular  walls  to  create 
micro  climates  for  growing  plants.  The  air  flow  through  that  space  was 
fiendishly  complex,  infinitely  more  so  than  that  inside  a  jet  engine.  One  of 
the  most  striking  features  was  the  tendency  for  some  bodies  of  air  to 
become  entrained  in  the  space  and  other  flows  to  meander  throughout  it 
for  long  periods  of  time  before  finding  the  exit.  This  experience  does  not 
bode  well  for  any  scheme  for  rapid  room  odorizing  and  reodorizing.  The 
room  odorizing  systems  that  have  been  done  cost  $.5M-$1M,  only  work 
with  a  single  odor  at  a  time,  and  still  only  claim  100  odor  changes  per 
hour. 

The  conclusion  of  this  investigation  was  that  it  would  be  impractical  to 
odorize  existing  CAVEs  if  rapid  odor  changes  were  desired.  However,  it 
would  be  possible  for  the  participant  to  wear  the  delivery  system 
designed  for  immersive  environments  and  to  provide  sufficient  ventilation 
to  immediately  dilute  and  to  quickly  evacuate  the  odors  after  they  are 


5 


introduced.  For  military  personnel,  the  delivery  system  would  be  a  small 
fraction  of  the  weight  of  the  field  pack  they  would  normally  carry. 

The  result  is  that  a  single  delivery  system  can  be  made  to  work  without 
too  much  compromise  in  the  three  primary  virtual  reality  formats. 

lil.B.  Capabilities 

The  final  delivery  device  [Photo  1]  had  the  following  attributes: 

•  portable 

•  wearable 

•  wireless 

•  10  odors 

•  simultaneous  display  of  multiple  odors 

•  600  seconds  of  continuous  display  of  each  odor,  enough  to  support  a  one 
hour  experience 

•  wide  range  of  orientations 

ill.C.  Requirements 

Much  of  the  effort  in  designing  the  final  delivery  system  went  into  the 
identification  of  the  requirements  and  the  testing  of  previous  design 
approaches  against  those  requirements.  Therefore  the  many  requirements 
for  the  odor  delivery  system  will  be  elaborated  upon  below.  These  fall  into 
the  following  categories: 

•  the  physical  requirements  of  a  portable  device. 

•  the  functional  requirements  of  an  odor  delivery  system  for  virtual 
reality  experiences. 

•  the  operational  requirements  of  a  system  which  must  work  in  an 
experimental  setting. 

•  the  organizational  requirements  of  such  a  system  in  order  for  it  to  be 
employed  by  the  end  user. 

Since  the  requirements  for  use  by  dismounted  personnel  are  the  most 
stringent,  they  will  be  the  ones  discussed  below. 

I11.C.1.  Physical  Requirements 

To  understand  the  requirement  that  the  delivery  system  must  work  in 
immersive  virtual  reality  applications,  it  is  necessary  to  make  the  Pi’s 
argument  about  the  advantages  of  such  systems. 
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In  spite  of  the  hype  surrounding  immersive  virtual  reality,  these  systems 
not  only  do  not  deliver  on  the  basic  promise  of  the  concept,  they  also  fail 
to  fully  implement  it.  That  this  omission  is  not  noted  more  urgently  has 
been  the  biggest  surprise  in  the  field’s  development. 

Immersive  systems  purport  to  allow  participants  to  naturally  interact 
with  virtual  environments  using  their  whole  bodies.  To  the  extent  that 
they  couple  head  movement  to  image  generation,  they  do  this  very  well. 
However,  the  BOOM™  provides  this  same  capability  with  better  visual 
images.  What  is  missing  is  the  ability  to  walk  naturally  around  the  virtual 
world  thus  engaging  the  entire  body  in  the  act  of  perception  which  is  how 
we  operate  in  the  real  world.  We  have  a  lifetime  invested  in  interpreting 
the  world  in  terms  of  the  height  of  our  eyes  from  the  floor,  the  length  of 
our  stride,  and  the  posture  of  our  body. 

A  very  few  experimental  systems  permit  a  person  to  walk  around  a  virtual 
world,  but  these  systems  are  still  connected  to  the  computer  by  wires. 
The  most  important  object  in  the  real  world,  the  wire,  is  invisible  in  the 
virtual  one  which  means  that  either  the  user  has  to  constantly  worry 
about  tripping  over  it  or  someone  in  the  real  world  has  to  handle  the  wire 
for  him.  A  truly  immersive  virtual  reality  system  would  be  completely 
untethered.  Wireless  head  tracking  and  wireless  virtual  image 
transmission  were  achieved  earlier  in  the  program.  What  remained  was 
wireless  odor  delivery.  Thus,  wireless  operation  is  a  requirement  of  the 
final  delivery  system. 

Ill.C.I.a.  Portable 

Wireless  operation  suggests  but  does  not  require  that  the  system  be 
portable.  By  itself,  portability  primarily  implies  a  weight  that  can  be 
carried  comfortably.  The  weight  requirement  established  at  the  beginning 
of  the  program  was  10  lbs  for  the  odor  delivery  capability  by  itself.  It 
was  assumed  that  the  soldier  of  the  future  would  be  geared  for  chemical 
warfare  and  would  have  external  power  and  clean  air  sources  that  could 
also  support  the  delivery  system.  The  weight  of  these  subsystems  is  not 
attributed  to  the  odor  delivery  system.  However,  it  is  presumed  that  low 
power  and  light  weight  air  systems  are  also  desirable  design  criteria. 

Ill.C.I.b.  Wearable 

The  system  must  be  packaged  in  a  way  that  makes  it  comfortable  to  carry. 


The  biggest  issue  here  was  where  the  delivery  system  should  be  worn — on 
the  back,  around  the  waist,  or  on  the  head.  On  the  head  would  imply  much 
lighter  weight,  shorter  tubing  and  perhaps  no  tubing,  and  therefore  no 
pressurized  air  system.  On  the  back  is  less  challenging  but  implies  a  tube 
to  the  head  which  can  be  flexed  as  the  head  turns. 

III.C.2.  Functional  Requirements 

In  addition,  there  are  the  requirements  imposed  by  the  desire  to  deliver 
odors  at  the  appropriate  moments  in  virtual  reality  simulations. 

lil.C.2.a.  Odors 

Up  to  ten  odors  must  be  available  for  display.  This  number  seemed  to  be 
sufficient  for  most  training  scenarios  that  we  considered.  Obviously,  more 
odors  would  provide  the  possibility  of  a  richer  experience,  but  it  is  a  rare 
day  when  a  person  can  remember  ten  odors  that  they  encountered  during  it. 
Time  will  tell  whether  a  significantly  larger  number  would  be  desirable. 
Besides,  with  the  current  odorant  repertoire,  it  is  unlikely  that  more  than 
ten  odorants  could  be  found  that  could  be  used  to  simulate  a  single 
situation. 

III.C.2.b.  Multi  Odor  Display 

The  odors  must  be  deliverable  either  singly  or  in  combination.  Most  odors 
will  be  used  one  at  a  time  because  people  are  not  very  good  at  identifying 
the  ingredients  when  even  a  small  number  of  odors  are  mixed.  However, 
when  exact  simulation  is  not  the  goal,  several  odorants  can  be  mixed 
together  to  represent  different  but  related  odor  sources.  For  instance, 
pine  smell  is  composed  of  a  few  primary  chemicals,  e.g.  a-pinene,  b- 
pinene,  d3-carene,  and  b-phellandrene.[Pine]  By  varying  the  proportions  of 
these  chemicals  a  variety  of  evergreen  plants  can  be  suggested.  Since  few 
modern  people  have  a  calibrated  knowledge  of  the  smells  of  particular 
species,  it  is  likely  that  no  one  will  notice.  Think  of  this  approach  as 
impressionistic  as  opposed  to  realistic  odor  painting. 

III.C.2.C.  Intensity  Control 

For  the  delivery  system  to  do  its  job,  it  is  not  enough  to  simply  display 
odor,  it  must  be  able  to  display  it  in  the  appropriate  strength  for  each 
moment  in  the  experience.  Several  of  our  earlier  delivery  systems  were 
completely  successful  at  displaying  many  odors,  but  were  totally  unable 
to  present  some  of  the  more  subtle  ones.  Betadine  which  is  used  as  an 
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antiseptic  prior  to  surgery  has  a  clear  smell  when  it  is  applied  to  a  large 
part  of  the  body,  but  the  odor  produced  by  passive  evaporation  of  this 
material  was  almost  undetectable. 

In  addition  to  being  able  to  present  the  odor  in  the  required  intensity,  it  is 
necessary  to  be  able  to  vary  that  intensity  as  the  participant  moves  closer 
to  or  away  from  the  odor  source  in  the  virtual  environment.  It  is  this 
coupling  of  odor  stimuli  with  physical  behavior  that  we  felt  was  the  most 
successful  aspect  of  odors  in  virtual  experiences.  Controlling  the  strength 
of  an  odor  by  head  movement  not  only  made  the  scene  seem  more  real,  but 
it  also  made  the  odor  itself  seem  more  real  and  more  believably  bound  to 
its  graphic  source. 

III.C.2.d.  Dynamic  Range 

There  is  another  reason  that  it  is  necessary  to  be  able  to  control  the 
intensity  of  an  odor.  People’s  sensitivity  to  odors  varies  enormously--by 
several  orders  of  magnitude.  Thus,  a  range  of  odors  that  is  perfectly 
obvious  to  one  person  may  be  imperceptible  to  another. 

It  is  possible  to  argue  that  these  Individual  differences  do  not  matter. 
What  matters  is  what  the  real  concentration  of  the  odors  are  in  the  real 
world  situation  to  be  simulated.  Ah,  but  how  do  we  know  what  that 
concentration  is?  We  have  no  instruments  for  measuring  odors  as  humans 
perceive  them.  In  fact,  we  really  have  no  instruments  for  measuring  odors 
at  all.  Instead,  we  have  instruments  for  measuring  chemicals.  The 
strength  of  odors  emanating  from  hog  farms  is  measured  by  panels  of 
people  who  smell  air  samples  diluted  in  measured  proportions  until  they 
cannot  smell  them  anymore.  Even  if  such  panels  could  be  afforded,  telling 
someone  that  they  do  not  have  a  good  enough  sense  of  smell  to  appreciate 
the  simulation  is  no  way  to  get  support  for  a  new  technology. 

A  person’s  sense  of  smell  is  roughly  exponential.  The  least  noticeable 
difference  is  about  1.1  times  the  current  concentration. [Smith]  Successive 
perceptions  based  on  head  movements  may  be  able  to  make  use  of  that  kind 
of  difference.  But  for  many  purposes  faint,  present,  and  strong 
concentrations  might  be  enough.  However,  most  technological  approaches 
for  delivery  systems  involve  linear  as  opposed  to  logarithmic  control  and 
apparently  impressive  linear  resolution  may  not  provide  enough  dynamic 
range  for  a  given  individual’s  experience.  Providing  the  same  perceptual 
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experience  for  individuals  with  very  different  odor  sensitivities  may 
require  an  extremely  sophisticated  delivery  system.  Varying  the 
concentrations  of  the  liquid  odorants  is  an  alternative  to  a  sophisticated 
delivery  system — albeit  a  cumbersome  one. 

Furthermore  given  that  the  final  system  is  the  only  variable 
concentration,  multiple-odor  delivery  system  available,  it  would  also  be 
desirable  if  it  could  be  used  for  studies  of  the  psychophysics  of  smell, 
studying  people’s  ability  to  experience  odors  in  the  laboratory.  Many  such 
experiments  can  be  imagined  and  the  results  of  some  of  these  would  have 
been  useful  to  have  in  designing  the  delivery  system  itself.  The  odor 
research  that  has  been  done  has  not  contemplated  applications  and  so  the 
questions  that  immediately  occur  to  someone  who  wants  to  exploit  odor 
effects  have  in  many  cases  not  been  asked,  let  alone  answered. 

III.C.2.e.  Consistency 

It  is  desirable  that  the  odor  concentrations  and  the  odors  themselves  be 
the  same  from  one  session  to  another  and  from  one  installation  to  another. 
This  requirement  is  more  demanding  than  it  seems  at  first  glance.  Much  of 
the  formal  research  that  has  been  done  has  used  peppermint  as  the 
odorant,  but  what  is  peppermint?  There  are  a  number  of  ways  of  producing 
this  odor  from  natural  materials  or  artificial  ones  and  the  kind  of 
peppermint  used  is  almost  never  specified  in  the  research  papers.  With 
traditional  techniques  for  producing  odorants  from  natural  materials 
there  is  almost  no  way  of  maintaining  precise  control  of  the  chemistry  of 
the  resulting  product.  Starting  with  pure  chemicals  gives  a  developer 
much  better  control  of  the  ingredients,  but  may  result  in  less  vivid  odors. 

III.C.2.f.  Speed 

While  speed  is  not  a  dimension  that  is  typically  associated  with  odors,  it 
is  relevant  in  this  application.  If  odors  are  to  be  experienced  as  part  of 
physical  behavior,  they  have  to  be  displayed  at  the  speed  of  that  behavior 
not  at  the  supposed  speed  of  the  sense  of  smell.  A  person  walking  at  four 
miles  per  hour  is  moving  70  inches  per  second  which  is  more  rapidly  than 
many  people  realize.  Odors  which  are  delivered  after  a  noticeable  lag  will 
not  be  associated  with  the  appropriate  sources  in  the  virtual  world.  Note 
that  this  argument  is  not  negated  by  the  fact  that  odor  perception  itself  is 
not  that  fast.  A  sensation  delayed  in  time  and  displaced  in  space  is 
incorrect.  Therefore,  very  fast  response  is  desirable.  Since  it  takes  about 
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1/20  second  for  the  air  to  be  transported  through  the  tubing  from  the 
delivery  system  to  the  nose,  that  is  the  lag  of  the  system  unless  it  is 
mounted  on  the  head. 

Speed  is  also  an  issue  in  that  pulse  width  modulation  is  the  technique  that 
we  use  to  control  the  intensity  of  the  odor. 

lll.C.2.g.  Odor  Duration 

The  delivery  system  must  contain  enough  odorant  material  to  support  a 
meaningful  training  experience.  Given  current  virtual  reality  technology, 
people  do  not  wear  a  head-mounted  display  for  more  than  an  hour.  During 
that  period,  there  needs  to  be  enough  odorant  to  display  odors  as  they  are 
encountered  in  the  experience. 

However,  since  people  start  to  habituate  to  an  odor  within  seconds  after 
they  encounter  it,  it  is  not  necessary  to  be  able  to  display  any  odor  for  the 
entire  duration,  because  the  participants  would  not  be  able  to  smell  it 
anyway.  (They  also  do  not  notice  when  an  odor  they  have  habituated  to  is 
removed.  That  is  a  non-event.)  Thus,  assuming  that  each  encounter  with  a 
new  odor  can  be  represented  by  a  few  seconds  of  odor  display,  the  amount 
of  odorant  required  can  be  modest.  One  hundred  seconds  of  display  per  odor 
is  probably  more  than  enough. 

III.C.2.h.  Quiet 

The  operation  of  the  delivery  system  cannot  distract  from  its  purpose. 
Therefore  any  noise  that  it  makes  when  operating  ideally  would  be  below 
the  threshold  of  human  perception  and  failing  that  could  be  rendered 
unnoticeable  by  sound  proofing  or  head  phones  or  masking  sounds.  There 
are  several  potential  sources  of  sound  in  a  delivery  system.  One  is  the 
sound  of  the  odor  selection  itself.  If  this  sound  is  noticeable,  the 
participant  always  receives  a  combined  auditory/olfactory  cue  which  is 
quite  unlike  anything  that  occurs  in  the  real  world  and  therefore 
disturbing. 

Another  potential  source  of  sound  is  from  the  pressurized  air  system  that 
will  be  required  if  the  odorized  air  must  travel  through  any  length  of 
tubing.  To  the  extent  that  such  a  sound  is  continuous  and  featureless,  it  is 
more  tolerable. 
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However,  pressurized  air  may  also  be  needed  to  drive  the  odorant 
materials  through  the  delivery  system.  If  this  system  involves  a  feedback 
loop  which  monitors  the  current  air  pressure  and  opens  a  valve  to  let  in 
more  air  when  it  falls  below  a  certain  pressure,  the  result  is  an 
intermittent  sound  which  is  much  more  disturbing  than  a  continuous  one. 
The  alternative  of  wearing  a  bottle  of  compressed  air  eliminates  this 
sound  at  the  cost  of  additional  weight. 

In  general,  the  fact  that  virtual  reality  systems  typically  include 
headphones,  and  audio  cues  generated  as  part  of  the  experiences  means 
that  the  sound  created  by  the  delivery  system  is  less  important  than  it 
would  be  otherwise. 

III.C.2.i.  Orientation  Insensitive 

Since  the  delivery  system  is  intended  to  be  worn  by  a  person  who  is 
physically  participating  in  a  virtual  world,  it  must  capable  of  operating 
during  that  behavior.  In  particular,  it  must  be  able  to  tolerate  the  range  of 
orientations  that  it  will  assume  as  the  participant  moves,  jumps,  bends 
over,  etc.  This  requirement  is  not  so  benign  given  that  it  is  a  liquid 
handling  system  which  typically  prefers  to  know  which  way  is  up  and 
which  down.  Indeed,  this  problem  surfaced  in  the  previous  system  which 
operated  well  during  use  and  only  had  a  problem  when  carelessly  placed  on 
its  side  on  the  bench  afterwards.  In  that  orientation,  the  liquid  odorants 
could  flow  against  the  pressurized  air  that  forced  them  through  the 
delivery  system  and  back  into  the  pumps.  While  the  worst  did  not  happen, 
the  hazard  was  clear. 

The  minimum  requirement  is  that  the  system  be  able  to  operate  during  45° 
tilts  which  are  the  worst  that  are  likely  to  occur  during  standing  and 
walking  behavior.  A  desirable  level  of  performance  is  for  the  system  to 
operate  during  90°  rotations  which  would  render  it  horizontal.  Such  a 
posture  is  the  worst  that  might  be  assumed  by  a  medic  leaning  over  a 
casualty  without  bending  his  knees,  but  it  might  be  quite  common  for 
infantrymen  crawling  along  the  ground.  Ideally,  the  system  would  continue 
to  operate  if  the  participant  stood  on  his  hands  or  laid  it  on  the  bench 
upside  down. 

III.C.2.j.  Contamination 

For  odors  to  be  most  effective,  they  must  be  displayed  against  a 
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background  of  no  odor.  Therefore,  it  is  important  that  the  only  odor  being 
perceived  is  the  odor  that  js  supposed  to  be  displayed.  Furthermore, 
because  we  so  completely  habituate  to  odors,  it  is  important  that  we  only 
smell  each  odor  when  it  is  supposed  to  be  displayed.  If  an  odor  is  present 
even  in  low  concentrations  in  between  intended  stimuli,  the  participant 
may  not  be  able  to  smell  it  at  all  when  it  is  displayed. 

There  are  a  number  of  places  where  such  contamination  can  occur  in  the 
delivery  system.  Storage  techniques  like  micro  encapsulation  and 
impregnated  porous  polymers  leak  odor.  Thus,  it  is  important  that  the 
storage  system  keep  each  odor  segregated  from  the  other  odors  and  from 
the  user  when  it  is  not  being  displayed.  It  can  also  adhere  to  the  tubing 
between  the  delivery  system  and  the  nose  and  be  re-emitted  over  time. 

The  brute  force  way  to  avoid  this  hazard  is  to  provide  separate  tubing  for 
each  odorant  all  the  way  to  the  nose.  Another  way  is  to  use  Teflon  tubing 
which  does  not  adsorb  much  odor  material.  Yet  another  way  is  to  heat  the 
tubing  all  the  way  to  the  nose. 

Contamination  of  the  tubing  has  not  been  observed  to  be  as  much  of  a 
problem  as  expected  with  this  delivery  system,  although  it  has  not  been 
tested  with  the  most  obnoxious  odorant  materials  such  as  benzyl 
mercaptan  which  can  be  smelled  through  a  gas  mask.  However,  one  source 
of  residual  odor  that  the  PI  had  not  thought  about  is  the  adhesion  of  strong 
odorants  in  high  concentration  to  nasal  hairs  which  causes  some  smells  to 
linger  after  the  delivery  system  has  stopped  displaying  them.  This 
phenomenon  can  also  occur  in  the  real  world. 

The  final  type  of  contamination  occurs  when  the  ambient  environment 
becomes  saturated  with  odors.  This  problem  can  be  avoided  by  filtering 
the  exhaled  air,  by  evacuating  it  with  a  ventilation  system,  or  by  having  a 
large  enough  volume  of  ambient  air  that  the  odorized  air  is  quickly  diluted 
after  it  is  exhaled.  Other  solutions  are  to  filter  the  ambient  air  before  it 
is  odorized  to  remove  any  contamination  or  to  use  bottled  air  which  will 
be  odorless. 

III.C.2.k.  Complete  Vaporization 

Contamination  can  also  occur  due  to  incomplete  vaporization  of  a  previous 
odorant  when  a  common  evaporation  chamber  is  used.  The  result  is  that 


the  previous  odorant  residue  is  vaporized  with  the  freshly  dispensed 
material  resulting  in  an  odor  mixture  rather  than  a  pure  smell. 

There  are  several  ways  that  incomplete  evaporation  can  occur.  If  the 
odorant  is  presented  to  the  vapor  transducer  faster  than  it  can  be 
vaporized,  the  odor  will  stiff  be  present  after  the  command  to  create  it  is 
turned  off.  The  easiest  way  to  correct  this  is  to  operate  the  vaporization 
transducer  a  bit  longer  than  the  odor  display  requires  to  burn  off  any 
excess  odorant.  If  desired,  the  extra  odor  can  be  shunted  into  a  filter 
instead  of  to  the  participant's  nose. 

III.C.2.1.  Vaporization 

Another  important  decision  is  the  choice  of  the  vaporization  technique  in 
the  first  place.  While  passive  evaporation  assures  that  the  odorant  is 
turned  into  a  vapor,  the  process  does  not  necessarily  produce  enough  odor 
for  a  vivid  experience.  In  particular,  subtle  odors  may  not  be  discernible 
at  all  even  if  they  are  quite  obvious  in  the  real  world.  In  addition,  the 
most  volatile  components  of  the  odorant  will  boil  off  first  leaving  the 
less  volatile  components  behind  and  the  odor  will  change  over  time. 

Therefore,  it  is  natural  to  consider  a  forced  vaporization  system.  The 
most  obvious  way  to  accelerate  evaporation  is  to  use  heat.  However,  while 
heat  will  speed  the  vaporization  process,  it  does  not  automatically 
eliminate  the  change  in  the  odor  that  occurs  as  ingredients  with  different 
vapor  pressures  are  boiled  off  one  at  a  time.  It  also  does  not 
automatically  assure  complete  vaporization  because  evaporation  is  an 
extremely  effective  means  of  cooling  and  care  must  be  taken  to  assure 
that  the  heater  is  not  itself  cooled  by  the  evaporation,  again  resulting  in 
incomplete  vaporization.  The  solution  to  this  problem  is  to  use  sufficient 
heat  at  a  temperature  somewhat  above  the  boiling  point  of  the  least 
volatile  component  of  the  odorant.  Enough  thermal  mass  must  be  available 
to  overwhelm  the  cooling  effect  of  the  vaporization  process. 

However,  there  are  reasons  to  be  wary  of  using  heat.  Some  of  the 
ingredients  of  an  odorant  may  chemically  react  with  each  other  when  heat 
is  applied.  Some  of  the  odorant  materials  have  low  flash  points  and  might 
ignite.  Finally,  heated  vapors  will  be  more  hazardous  that  cool  vapors.  For 
these  safety  reasons,  heat  was  ruled  out  as  a  means  of  vaporizing  the 
odorants. 
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Whatever  the  means  of  vaporization  chosen,  it  is  preferable  that  it 
completely  vaporize  the  material.  It  is  possible  to  dispense  droplets  and 
allow  them  to  vaporize  in  the  air  stream.  However,  if  the  droplets  are 
large,  they  may  settle  against  the  walls  of  the  tubing.  If  they  are  smaller 
than  5  microns,  they  are  considered  a  breathable  mist  and  are  less  likely 
to  stick  to  a  surface. 

III.C.2.m.  Safety 

The  delivery  system  must  be  safe  for  the  user  to  wear.  Many  of  the  raw 
chemicals  used  as  odorants  are  toxic,  carcinogenic,  sensitizing,  etc.  as 
they  are  sold  in  neat  (pure)  form.  At  the  same  time,  they  are  meant  to  be 
breathed,  applied  to  the  skin,  and  in  many  cases  ingested  when  they  are 
used  as  food  flavorings.  The  difference  is  that  they  are  typically  diluted 
heavily  before  they  are  used.  Unfortunately,  the  safe  levels  of  dilution  are 
not  published  anywhere.  In  addition,  many  of  the  traditional  materials  that 
are  extracted  from  natural  plant  and  animal  products  have  never  been 
studied  for  their  safety. 

In  any  of  our  delivery  system  designs,  the  raw  chemical  is  diluted  from  by 
a  factor  of  10  to  1000  in  a  diluent  such  as  water,  alcohol,  or  mineral  oil. 
This  liquid  is  then  further  diluted  by  the  air  stream  into  which  it  is 
vaporized.  (100  nanoliters  of  liquid  are  evaporated  per  millisecond  into 
.16  milliliters  of  air  for  a  maximum  1600:1  ratio.  However,  since  the 
typical  duty  cycle  of  the  odor  delivery  is  likely  to  be  around  1  millisecond 
on  per  20  milliseconds  off,  another  20:1  dilution  is  occurring  for  a 
32000:1  total  dilution.  Further  dilution  occurs  at  the  nose  where  ambient 
air  is  mixed  with  the  odorized  air  as  it  is  breathed. 

It  is  possible  that  a  vaporized  odorant  could  condense  on  a  cool  surface 
either  on  the  device  or  on  a  nearby  surface.  It  could  then  dry  out  leaving 
the  concentrated  chemical.  A  person  touching  that  residue  might 
inadvertently  rub  their  eyes  or  touch  their  lips.  While  we  have  not 
experienced  any  such  problems  with  the  current  device,  earlier  efforts 
were  visited  by  this  hazard.  Therefore,  it  is  necessary  that  operating  the 
device,  loading  it  for  use,  and  cleaning  it  after  use  do  not  lead  to  the 
contamination  of  the  device  itself  or  to  a  hazard  for  the  user. 

Another  safety  issue  arises  when  mounting  the  delivery  system  on  the 
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head  is  attempted.  Doing  this  is  attractive  because  the  tubing  can  be 
dispensed  with,  eliminating  the  need  for  the  pressurized  air  and  therefore 
the  pumps  and  batteries  that  generate  it  or  the  heavy  air  bottle  strapped 
to  the  thigh  that  could  contain  it.  The  possibility  of  releasing  toxic 
chemicals  into  the  nose  has  been  mentioned.  In  addition,  piezo  systems 
require  special  power  supplies  that  operate  at  high  voltages  and  create 
transients  at  even  higher  voltages.  In  the  current  climate  in  which  people 
are  concerned  about  their  cell  phones,  the  combination  of  high  voltages 
and  high  frequencies  would  be  sure  to  set  off  alarms  whether  or  not  the 
hazard  was  real. 

III.C.3.  Ease  of  Use 

Ideallly,  the  hardware  should  be  easy  to  turn  on.  The  components  should  be 
easy  to  connect.  And  software  should  alert  the  user  to  the  health  of  the 
system.  The  software  should  make  odor  selection,  stimulus  triggering,  and 
concentration  control  easy  for  developers  to  specify.  Test  procedures 
should  be  available  that  allow  personnel  to  put  the  device  through  its 
paces  when  it  is  on  the  bench  so  a  participant  is  not  required  during  the 
testing  process. 

In  any  setting  the  easier  the  device  is  to  use,  the  more  likely  that  it  will 
be  used  In  an  end  user  site,  almost  no  operational  hassle  or  hazard  will  be 
tolerated.  Ideally,  the  device  would  be  maintenance  free  and  no  harder  to 
reload  than  an  ink  jet  printer.  Unfortunately,  we  cannot  claim  that 
anything  like  that  ease  of  use  has  been  achieved.  Ink  jet  printers  have 
probably  had  over  a  billion  dollars  invested  in  their  incremental 
improvement  over  a  20  year  period  to  achieve  their  current  level  of 
performance. 

Just  purchasing  the  chemicals  to  be  used  as  odorants  was  a  major 
research  project  for  the  PI.  The  companies  would  not  sell  directly  to  him 
because  ARC  is  not  a  chemical  laboratory.  After  months  of  playing  Hamlet, 
the  University  of  Connecticut  decided  that  it  was  completely  spooked  by 
the  thought  of  purchasing  these  chemicals  and  did  not  feel  that  students 
could  be  asked  to  handle  them  at  all  or  to  perform  the  simple  task  of 
diluting  them  to  safe  concentrations.  One  local  chemical  lab  was 
approached  and  after  first  agreeing  ultimately  declined.  A  second  was 
finally  convinced  to  do  the  work. 
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Ease  of  use  implies  several  levels  of  infrastructure  that  do  not  exist.  The 
odorants  need  to  be  diluted  to  a  safe  level.  They  need  to  be  packaged  into 
easily  loadable  cartridges  that  contain  all  of  the  odorants  required  for  a 
particular  virtual  reality  scenario.  And  they  need  to  be  quickly  available  in 
that  form  to  whoever  needs  them  without  special  facilities  or  trained 
personnel  intervening.  Given  that  end  users  will  want  to  create  their  own 
scenarios  and  that  standard  scenarios  that  would  be  employed  by  huge 
numbers  of  systems  are  not  likely  to  exist  for  some  time,  a  programmable 
cartridge  that  can  be  easily  loaded  with  the  desired  odorants  would  need 
to  be  developed  first.  This  could  be  loaded  by  the  end  user  or  probably 
could  be  more  easily  done  by  a  internet-based  service  with  overnight 
delivery.  That  at  least  would  permit  the  packaging  to  be  less 
sophisticated  and  would  centralize  the  facilities,  skills,  and  chemical 
stocks  needed  to  load  the  cartridges.  Even  so,  a  great  deal  of  effort  would 
have  to  be  spent  on  these  packaging  and  infrastructure  needs  before  the 
delivery  technology  would  be  the  limiting  factor.  That  implies  a  market 
for  such  materials  that  is  unlikely  to  develop  for  some  time.  More  likely  is 
that  an  arcade  unit  will  be  mass  produced  for  a  particular  game  that  will 
have  a  short  life,  and  odor  delivery  will  then  fizzle  for  some  time  before 
the  needed  technology  is  developed  for  some  other  purpose,  e.g.  drug 
delivery. 

III.C.4.  Maintainable 

In  addition  to  being  easy  to  use,  the  delivery  device  needs  to  be  easy  to 
maintain  or  be  maintenance  free.  The  importance  of  this  issue  cannot  be 
stressed  too  much.  Over  and  over  we  found  that  miniature  components 
could  be  fouled  by  miniature  hazards  and  that  what  seemed  to  be 
sufficient  precautions  would  be  thwarted  by  some  new  level  of 
complication. 

Ink  jet  printers  are  the  only  examples  of  precision  liquid  handling  devices 
that  are  widely  available  at  low  cost.  They  include  a  mature  set  of 
techniques  and  technologies  for  making  them  relatively  maintenance  free. 
The  ink  cartridges  are  carefully  engineered  to  operate  with  one  kind  of  ink 
which  itself  has  been  formulated  by  legions  of  PhD  chemists  to  flow 
properly  in  that  exact  cartridge  and  that  exact  print  head.  The  printer 
software  contains  elaborate  procedures  for  clearing  the  ink  lines  and 
wiping  the  print  heads  before  it  starts  to  print.  Even  so,  HP  has  found  it 
expedient  to  make  its  print  heads  disposable  rather  than  to  keep  them 
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operating  for  long  periods  of  time. 

The  odor  delivery  system  in  contrast  must  operate  with  a  wide  and 
unbounded  range  of  toxic  and  corrosive  chemicals  mixed  in  a  number  of 
different  diluents  with  different  physical  properties,  including  water, 
alcohol,  and  mineral  oil.  The  physical  properties  of  the  individual 
chemicals  are  not  readily  available  and  seem  to  be  treated  as  trade 
secrets  in  the  industry.  In  addition,  because  of  the  volatility  of  the  most 
of  the  diluents,  it  is  likely  that  they  will  dry  up  inside  the  delivery 
system  if  it  is  not  used  for  a  period  of  time.  The  solid  residue  remaining 
can  clog  the  tubing  and  the  valves  permanently.  Therefore,  it  is  necessary 
for  the  delivery  system  to  not  only  be  maintainable  when  it  is  in  constant 
use,  but  it  also  must  tolerate  periods  of  disuse  without  complaint. 

It  is  also  desirable  for  the  loading  and  unloading  of  odorants  not  to  be  a 
messy  procedure.  Early  versions  of  the  system  could  drip  when  odorant 
cartridges  were  inserted  or  removed.  The  current  system  does  not  have 
this  problem. 

III.D.  Previous  Designs 

The  final  design  was  preceded  by  a  considerable  number  of  experiments 
and  by  several  complete  systems.  Passive  evaporation,  heat-forced 
evaporation,  electrostatically  induced  evaporation,  micro  encapsulation, 
odor  impregnated  polymers,  micro  venturi  mixing,  oscillating  tubes  to 
create  droplets,  ink  jet  print  heads,  and  three  different  piezo-induced 
vaporization  techniques  were  all  tested.  Since  these  approaches  were 
described  in  earlier  reports  they  do  not  need  to  be  detailed  here.  Only  the 
testing  of  the  previous  system  which  was  accomplished  in  the  final  year 
and  the  earlier  attempts  to  use  piezo-induced  vaporization  will  be 
elaborated  upon  further. 

III.D.1.  Micro  Venturi  System 

The  most  recent  complete  system  was  also  capable  of  wearable  and 
wireless  operation.  It  also  provided  the  rapid  response  and  ability  to 
control  the  concentration  needed  for  virtual  reality  experiences.  Tiny 
amounts  of  liquid  odorant  were  released  into  a  micro  venturi  chamber 
where  they  were  subjected  to  maximal  turbulence  and  therefore  maximal 
mixing  with  the  air  stream  that  was  flowing  through  the  chamber.  The 
liquid  was  released  through  a  tiny  jet  into  the  outlet  of  the  venturi  where 
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it  starts  to  widen.  Six  jets  each  delivering  a  different  odorant  were 
spaced  around  the  outlets  of  each  of  two  venturis  to  provide  12  odors 
total.  [Photo  2] 

The  odorants  were  stored  in  syringes  whose  plungers  had  been  removed 
and  a  pressurized  air  system  substituted.  The  needle  of  each  syringe  was 
inserted  into  a  tube  which  ran  through  a  valve  which  controlled  the  flow 
of  odorant  into  the  venturi  chamber.  Using  air  pressure  to  force  the 
odorants  through  the  valves  worked  better  than  previous  methods  that  had 
used  mechanical  action.  While  the  previous  techniques  had  worked,  it  was 
difficult  to  assure  that  the  movement  was  uniformly  smooth.  Instead,  the 
mechanism  would  occasionally  stick  slightly  before  continuing  to  move. 

III.D.I.a.  Venturi  Operation 

This  system  worked  well  for  periods  of  time,  but  occasionally  incomplete 
evaporation  was  observed  when  the  venturi  was  overloaded,  and  there  was 
a  tendency  for  the  odor  vapor  to  condense  downstream  from  the  venturi.  In 
addition,  there  was  a  tendency  for  the  valves  to  clog  from  particles  in  the 
odorants  or  from  residue  from  odorants  that  evaporated  when  the  system 
was  not  in  use.  While  the  clogging  was  not  the  fault  of  the  venturi  design, 
it  did  necessitate  a  redesign  of  the  rest  of  the  system  and  the  whole 
venturi  approach  was  a  casualty  of  that  process. 

The  venturi  system  was  quite  tolerant  of  the  changes  in  orientation  that 
occurred  during  normal  walking  and  stooping  behavior  as  might  be 
expected  of  a  medic  leaning  over  a  battlefield  casualty.  However,  a 
catastrophic  problem  was  observed  one  day  when  it  was  laid  on  its  side 
instead  of  on  its  base  on  the  bench  after  use.  In  spite  of  the  pressurized 
air  being  applied  to  the  odor  reservoirs,  the  liquid  odorant  was  able  to 
flow  out  of  the  reservoirs  and  back  into  the  tubing  towards  the  pumps  that 
were  generating  the  pressure.  While  the  pumps  were  not  damaged  and  the 
problem  could  be  avoided  by  being  more  careful  to  keep  the  device  properly 
oriented,  the  potential  consequences  of  a  mistake  were  sufficiently 
severe  that  the  reservoirs  had  to  be  redesigned. 

III. D.I.b.  Clogging 

The  most  severe  problem  was  the  clogging  of  the  tubing  and  valves  that 
could  occur  when  a  microscopic  particle  would  become  lodged  in  a  valve 
requiring  it  to  be  replaced.  Related  to  that  was  the  tendency  of  the 
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original  valves  to  be  corroded  by  some  of  the  odorant  chemicals.  The 
latter  problem  was  addressed  by  switching  to  valves  with  a  Viton  coating 
which  were  more  expensive  ($150)  than  the  ones  we  had  been  using  ($100). 

The  clogging  problem  was  more  difficult.  We  built  an  automatic  flushing 
system  which  cleared  the  odorants  from  the  valves  by  running  alcohol 
through  them  after  use.  However,  this  system  also  used  valves  and  there 
was  always  some  odorant  remaining  between  the  fork  through  which  the 
flushing  material  was  inserted  and  the  odor  reservoirs.  This  material 
could  dry  up  and  clog  the  tubing  if  not  the  valves.  It  was  finally  decided  to 
give  up  on  automatic  cleaning  and  instead  to  prevent  clogging  by  more 
careful  handling  of  the  odorant  materials  both  in  and  out  of  the  system, 
and  by  completely  flushing  the  system  manually  after  every  use.  The  fact 
that  this  procedure  violates  the  ease  of  use  requirement  is  not  lost  on  the 
PI,  but  preventing  the  destruction  of  the  valves  was  more  important. 

Prevention  was  accomplished  by  passing  the  odorants  through  a  2  micron 
filter  during  the  loading  of  the  reservoirs.  In  addition,  a  10  micron  filter 
was  placed  in  the  odorant  stream  before  each  valve.  Without  prefiltering 
through  the  2  micron  filter,  the  10  micron  filter  became  clogged. 

Flushing  was  accomplished  by  disconnecting  the  odorant  reservoirs  and 
running  alcohol  through  the  system  and  then  running  air  through  it.  This 
was  done  for  each  odor  channel  in  turn.  The  entire  procedure  took  only  a 
few  minutes,  but  would  be  likely  to  be  considered  a  hassle  by  end  users. 

III.D.2.  Piezo-Induced  Vaporization 

There  were  enough  problems  with  the  micro  venturi  chambers  that  we 
revisited  the  fundamental  issue  of  how  to  transform  the  liquid  odorant 
into  a  vapor. 

III.D.2.a.  Piezo  Crystal 

One  approach  that  had  been  considered  repeatedly  was  the  use  of  a  piezo 
crystal  to  mechanically  rupture  the  liquid  to  create  droplets  so  small  that 
they  could  be  considered  a  vapor  or  would  evaporate  as  they  passed 
through  the  tubing  to  the  nose. 

The  initial  experiments  were  conducted  by  Dr.  Thomas  Avedisian  in  the 
Mechanical  Engineering  department  at  Cornell.  By  applying  a  10  Khz  signal 
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to  a  crystal  with  a  drop  of  liquid  deposited  on  it,  he  succeeded  in 
producing  droplets,  but  they  were  too  large  and  flew  off  in  all  directions 
wetting  everything  in  sight. 

III.D.2.b.  Piezo  Film 

Next,  Dr.  Martin  Fox  in  the  Electrical  Engineering  department  at  the 
University  of  Connecticut  tested  a  piezo  film  with  a  10  Khz  signal.  The 

droplets  it  generated  were  nearly  invisible.  The  result  was  a  faint  but 

detectable  odor  that  was  not  strong  enough  for  our  purposes.  It  was  not 
clear  how  to  apply  the  liquid  odorant  to  the  film.  It  appeared  that  it  would 
be  necessary  to  release  odorant  droplets  onto  the  film.  That  technique 
would  not  be  orientation  independent  because  the  droplets  would  not  fall 
in  the  correct  direction  if  the  device  was  tilted.  Moreover,  the  machinery 
required  to  release  the  droplets  seemed  to  be  as  difficult  as  the 

vaporization  problem. 

For  that  reason,  the  venturi  approach  seemed  attractive  because  no 
droplet  formation  was  required  and  such  a  device  would  work  in  any 
orientation.  (It  was  the  reservoirs  not  the  venturi  that  eventually  caused 
problems  in  extreme  orientations.)  Initially,  the  piezo  film  seemed  to 

offer  the  possibility  of  fabricating  a  number  of  different  mechanical 
components  out  of  the  same  material,  but  after  initially  expressing  great 
interest,  the  manufacturer  failed  to  deliver  any  manufacturing 
information  beyond  that  included  in  the  kit  they  sold  us.  We  considered 
trying  to  wick  the  odorant  onto  the  piezo  film  by  capillary  action  by 
having  second  film  a  slight  distance  from  it.  The  vibration  of  the  film 
would  then  cause  some  of  the  material  to  vaporize  and  more  liquid  would 
fill  the  space  it  had  occupied.  Such  wicking  is  used  in  some  ink  jet 
printers  which  are  connected  to  the  ink  cartridges  by  a  fine  sponge. 
However,  unless  we  could  assure  that  no  odorant  evaporated  while  the 
piezo  was  not  vibrating,  we  would  need  valves  to  prevent  the  vapor  from 
escaping  into  the  air  stream.  Those  valves  added  a  level  of  complexity  and 
cost  that  appeared  to  nullify  the  advantages  of  the  approach. 

III.D.2.C.  Piezo  Sprayer 

A  third  ultrasonic  technique  was  a  spray  nozzle  that  could  be  used  for 
applying  paint  or  insecticides.  In  this  case,  a  pair  of  piezo  crystals 
surrounded  a  metal  tube  which  was  the  vibrating  element.  The  system 
operated  at  100  Khz  and  generated  enormous  amounts  of  vapor.  Given  the 
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exotic  materials  used  to  create  this  device,  we  did  not  know  how  well  it 
would  scale  down  and  how  to  go  about  designing  a  low  capacity  version. 

ilI.E.  Final  Approach 

The  final  approach  was  to  appropriate  the  piezo  crystal  from  a  nebulizer, 
which  is  device  that  sits  at  the  bottom  of  a  volume  of  water  and 
generates  water  vapor  at  a  prodigious  rate.  This  crystal  operates  at  1.2 
Mhz.  Its  principles  of  operation  are  outlined  in  Figure  1:  Functional  Flow 
Diagram  at  the  end  of  the  report. 

III.E.1.  Cooling  the  Crystal 

Since  it  is  submerged  in  water  during  its  intended  operation,  it  is  cooled 
by  the  water  itself.  When  we  tested  it  with  odorant  material,  it  was  quite 
successful.  It  vaporized  the  liquid  rapidly  and  completely.  But  there  were 
two  problems.  First,  without  the  water  to  cool  it,  the  crystal  overheated 
and  failed.  Second,  some  of  the  odorant  materials  reacted  with  the 
crystal,  discoloring  it  and  leaving  us  to  ponder  the  lethality  of  the  newly 
created  mystery  materials. 

The  overheating  was  solved  by  considering  the  crystal  as  part  of  the 
larger  system.  The  vaporized  odorant  wad  mixed  with  the  5  liter  per 
minute  air  stream  generated  by  a  pump  or  released  from  an  air  bottle  that 
forced  the  odorized  air  through  the  tubing  to  the  nose.  Running  this  air 
stream  over  the  crystal  cooled  it.  On  the  bench,  the  crystal  had  failed 
because  there  was  no  forced  air  running  across  it.  Using  the  air  stream  for 
cooling  was  initially  resisted  because  we  thought  it  might  interfere  with 
the  operation  of  the  crystal  or  it  might  blow  unevaporated  liquid  down  the 
tube,  but  once  we  overcame  that  prejudice  it  worked  well. 

III.E.2.  Coating  the  Crystal 

The  second  problem  was  the  reaction  of  the  crystal  with  some  of  the 
odorant  materials.  We  experimented  with  a  number  of  techniques  including 
coating  the  crystal  with  Teflon.  Initial  coatings  were  too  thick  and 
damped  the  oscillation  of  the  crystal.  After  some  experimentation,  we 
were  able  to  determine  a  thickness  that  would  protect  the  crystal  while 
still  allowing  it  to  work. 

III.E.3.  Orientation  Tolerant  Vaporization 

The  remaining  problem  was  presenting  the  liquid  odorant  to  the  crystal  In 
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a  way  that  would  work  in  a  wide  range  of  orientations.  As  pointed  out 
earlier,  droplets  will  not  always  fall  onto  the  crystal  if  the  apparatus  has 
been  tilted  away  from  vertical.  Another  problem  is  that  unless  they  are 
forced  out  of  a  small  aperture  at  very  high  pressure,  droplets  will  not  fall 
from  the  orifice  until  they  reach  a  critical  mass  sufficient  to  overcome 
the  surface  tension  and  viscosity  of  the  material  as  they  relate  to  the 
size  and  geometry  of  the  orifice.  If  a  droplet  forms  that  is  not  of 
sufficient  weight  to  break  loose  and  the  apparatus  rotates,  the  liquid  will 
flow  around  the  orifice,  perhaps  falling  or  dripping  in  the  wrong  direction. 

The  solution  to  this  problem  was  to  use  the  viscosity  and  surface  tension 
of  the  liquid  odorant  to  our  advantage.  The  tendency  of  a  partial  droplet  to 
form  at  the  end  of  the  tubing  was  such  that  it  would  bulge  out  from  the 
end  of  the  tubing  little  by  little  and  until  it  reached  a  critical  mass  it  was 
held  in  place  protruding  from  the  tube  without  any  tendency  to  be  shifted 
out  of  that  orientation  by  tilting  of  the  apparatus.  Rather  than  forcing  a 
complete  droplet  out  of  the  orifice  to  fall  onto  the  crystal  by  gravity,  the 
bulging  of  the  partial  droplet  was  allowed  to  expand  until  it  bridged  the 
tiny  gap  between  the  orifice  and  the  crystal.  When  the  bulge  made  contact 
with  the  crystal,  the  material  touching  the  crystal  was  vaporized 
instantaneously. 

By  this  means  a  partial  droplet  much  smaller  that  what  would  be  required 
to  break  loose  by  gravity  can  be  delivered  over  a  wide  range  of 
orientations.  It  is  conceivable  that  the  bulge  would  flow  down  the  outside 
of  the  tube  if  the  device  is  completely  inverted,  but  participants  are  not 
expected  to  do  hand  stands  or  other  gymnastics  in  virtual  reality. 

III.E.4.  Orientation  Tolerant  Reservoirs 

As  mentioned  earlier,  the  potential  consequences  of  laying  the  delivery 
device  down  incorrectly  when  it  was  not  in  use  were  so  great  that,  even 
though  the  worst  never  happened,  we  felt  that  that  hazard  should  be 
diminished.  In  addition,  the  usefulness  of  the  device  would  be  increased  if 
the  range  of  operating  orientations  could  be  increased.  Finally,  the  fact 
that  there  was  a  change  in  shape  and  dimension  from  the  reservoir  to  the 
tubing  meant  that  it  was  possible  for  the  air  to  get  past  some  of  the 
odorant,  thereby  stranding  it. 

The  solution  we  chose  was  to  use  the  tubing  itself  as  the  reservoir.  When 
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the  liquid  is  in  the  tubing  with  no  air  pressure  being  applied,  it  stays  in 
place  by  capillary  action  or  by  surface  tension  regardless  of  how  the 
orientation  is  changed.  In  addition,  since  the  tubing  used  as  reservoir  and 
the  tubing  used  as  a  conduit  look  the  same  to  the  liquid  odorant,  its 
behavior  is  the  same  throughout  the  system.  As  before,  pressurized  air  is 
applied  to  each  reservoir  and  when  the  valve  in  the  corresponding  channel 
opens,  the  liquid  is  propelled  through  the  tubing  onto  the  piezo  crystal 
which  vaporizes  it. 

III.E.5.  Delivery  System  Elements 

Now  that  the  requirements  have  been  discussed  and  the  principles  of 
operation  of  the  final  device  have  been  detailed,  we  can  run  through  the 
device  itself  and  the  procedures  for  operating  it. 

III.E.6.  Functional 

The  device  delivers  from  1  to  10  odors  with  a  10  millisecond  data 
transmission  time,  a  1  millisecond  electromechanical  response  time,  and 
a  40  millisecond  air  transport  time  to  the  nose.  It  provides  control  of 
concentration  over  a  100:1  range.  It  provides  up  to  600  seconds  of 
continuous  odor  delivery.  However,  since  people  quickly  habituate  to  odors 
and  no  longer  smell  them  after  a  few  seconds,  it  is  possible  and 
preferable  from  a  safety  point  of  view  to  turn  off  an  odor  which  is 
supposed  to  be  present  but  is  no  longer  perceived  without  the  participant 
noticing  its  withdrawal.  Therefore,  the  600  seconds  is  more  than 
sufficient  to  support  a  1/2  hour  to  1  hour  experience. 

III.E.7.  Physical 

The  device  is  contained  in  a  clear  plastic  cylinder  6.5  inches  in  diameter 
and  11  inches  high  weighing  7.5  pounds.  [Photo  1]  The  unit  comprises  the 
following  components: 

•  a  7  liter  per  minute  air  pump  for  driving  the  main  air  stream  through  the 

tubing 

•  a  10  psi  air  pump  for  driving  the  liquid  odorants  through  the  10  odor 

delivery  channels 

*  alternatively,  there  can  be  a  17  pound  air  bottle  with  1/2  hour  of 

air  strapped  to  the  participant’s  thigh.  It  is  silent  and  with 

*  two  miniature  regulators  can  replace  both  pumps  and  their 

batteries. 

•  three  external  12  volt  batteries  worn  on  a  belt  (for  weight  distribution) 
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which  provide  36  volts  for  running  the  piezo  as  well  as  power  to 
non  odor  delivery  devices  such  as  the  HMD  and  the  tracking  target. 

These  weigh  6  pounds. 

•  one  surface-mount  printed-circuit  board  with  the  following  subsystems 
*RF  transceiver 

*RF  address  decoding  and  error  checking 

*RF  antenna  for  receiving  from  and  transmitting  to  the  host  PC 

*a  40  Mhz  8051  micro  controller  with  the  following  software: 

-wireless  packet  control  software 

-10  channels  of  pulse  width  modulation  for  the  valves 

-cleaning  procedure  software 

-wireless  status  software  for  reporting  back  to  the  PC 

*  a  high  voltage  piezo  drive  circuit 

*  10  channels  of  valve  control  electronics 

*  an  air  reservoir  pressure  sensor 

*  an  air  reservoir  inlet  valve 

•  10  each  2  cc  odorant  reservoirs 

•  an  air  manifold  for  distributing  the  air  pressure  to  the  10  reservoirs 

•  10  odorant  channels  terminating  a  small  distance  from  the  piezo  crystal 

•  a  1.2  Mhz  piezo  crystal  for  vaporizing  the  odorant 

•  a  tube  for  delivering  the  odorized  air  to  the  nose 

•  an  external  PC  controls  the  device  and  communicates  with  it  through  a 

serial  port  and  a  wireless  transceiver,  high  speed  ethernet  link.  The 
simulation  machine  could  directly  control  the  delivery  system,  but  it  is 
best  not  give  too  many  real  time  functions  to  the  same  computer. 
Currently,  the  delivery  system  is  running  under  Windows,  the  tracking 
under  LINUX,  and  the  graphics  under  IRIX.  None  of  these  operating 
systems  are  ideal  or  good  or  even  acceptable  for  this  kind  of  function. 
The  ideal  would  be  to  strip  LINUX  of  functionality  to  the  point 
where  it  had  an  old  fashioned  Load-and-Go  capability  so  that  there  was 
no  runtime  operating  system  at  all,  just  the  application  code.  This  way 
the  developer  could  work  with  hard  timing  of  the  kind  that  you  have 
with  hardware,  not  the  probabilistic  timing  so  beloved  of  academics 
that  is  foisted  on  developers  by  priority  interrupt  systems.  The  Pi’s 
own  systems  have  always  worked  been  based  on  hard  timing  but  it  is 
harder  and  harder  to  get  past  the  operating  systems  that  are  bundled 
with  computers  to  look  at  what  it  takes  to  drive  the  underlying 
hardware  and  to  write  the  software  to  do  it.  LINUX  at  least  provides  the 
source  code. 
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III.E.8.  Flow  Through  the  Odorant  System 

The  pressurized  air  from  the  reservoir  pump  is  distributed  to  each  of  the 
reservoirs  through  the  air  manifold  at  the  top  of  the  delivery  system.  Each 
reservoir  is  a  2”  cylinder  with  20  inches  of  1/8”  ID  tubing  coiled  inside  of 
it  and  plugs  into  the  manifold  from  below.  Special  fittings  connect  the 
reservoir  to  the  manifold  and  prevent  odorant  from  dripping  from  either 
the  reservoir  or  the  manifold  when  the  former  is  unplugged  from  the 
latter.  When  pressure  is  applied  to  the  reservoir,  the  liquid  odorant  is 
driven  from  the  reservoir  up  through  the  connecting  fitting  in  the  manifold 
and  from  there  through  a  10  micron  filter  that  protects  the  miniature 
valves  from  any  particles  that  may  have  gotten  into  the  odorant.  The 
filtered  odorant  is  forced  against  the  valve  input  at  10  psi.  When  the  valve 
opens  for  as  little  as  a  millisecond,  a  tiny  amount  of  odorant  is  forced 
through  rigid  tubing  that  is  positioned  so  that  its  end  is  a  tiny  distance 
from  the  piezo  crystal. 

The  manifold  is  round  and  the  reservoirs  are  evenly  spaced  around  it.  The 
valves  are  also  arranged  in  a  ring  and  the  rigid  tubing  ends  in  a  circle  of 
orifices  that  are  spaced  around  the  circular  piezo  crystal.  It  is  the  shape 
of  the  crystal  that  dictates  the  shape  for  the  rest  of  the  system. 

The  5  liter  per  minute  air  flow  is  blowing  continuously  over  the  Teflon 
coated  crystal  simultaneously  keeping  it  cool  and  removing  the  odorant 
vapor  to  be  delivered  to  the  nose.  5  liters  per  minute  is  also  the  amount  of 
air  required  for  sedentary  human  breathing  if  the  system  provides  all  of 
the  air  the  participant  is  breathing.  However,  since  additional  ambient  air 
is  breathed  along  with  the  odorized  air,  there  is  enough  air  to  support  the 
exertion  of  walking  as  opposed  to  sitting. 

III.E.9.  Control 

Control  is  currently  assumed  to  originate  from  the  host  processor  which 
keeps  track  of  the  participant’s  current  location,  the  participant’s  current 
action,  and  the  location  and  of  odors  and  potential  odor  sources  in  the 
simulated  environment.  The  host  computer  then  directs  the  PC  to  command 
the  delivery  device  to  release  the  appropriate  odor  when  the  participant  is 
near  the  virtual  odor  source. 

Actually,  several  steps  are  being  glossed  over  here.  The  host  machine 
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sends  a  high  speed  ethernet  message  directing  the  PC.  The  PC  in  turn 
directs  its  RF  software  to  transmit  a  command  to  the  delivery  device.  The 
RF  software  builds  a  packet  that  is  sent  three  times  at  50  Khz.  This 
packet  contains  an  address  of  the  the  receiver  to  keep  it  from  responding 
to  spurious  transmissions  at  the  same  frequency.  The  packet  is  sent  three 
times  so  that  successive  messages  can  be  compared  to  see  if  what  has 
been  received  is  indeed  a  valid  message.  Once  the  packets  have  been  built, 
they  are  sent  through  the  serial  interface  through  the  RF  transmitter  to 
the  receiver  on  the  delivery  device. 

The  delivery  device  has  its  own  microcontroller  which  is  running  the 
packet  decoding  software  which  decides  if  the  received  packet  is  indeed  a 
valid  message.  If  it  is,  the  microcontroller  interprets  it  to  see  which 
valves  are  being  addressed.  It  then  determines  the  duty  cycle  of  the  pulse 
width  modulation  that  has  been  specified  to  control  the  concentration  of 
the  odor.  This  is  specified  in  terms  of  milliseconds  on  versus  milliseconds 
off.  Then,  the  number  of  on/off  cycles  before  the  odor  burst  is  terminated 
is  specified.  Since  the  length  of  period  can  be  any  number  of  milliseconds 
and  the  ratio  of  on  time  to  off  time  is  completely  programmable  within 
that,  there  are  many  more  possible  intensities  beyond  the  100:1  range  that 
the  requirement  specified. 

There  is  additional  software  in  the  microcontroller  to  report  the  status  of 
the  delivery  device  back  to  the  PC.  This  is  not  done  on  a  regular  basis 
because  it  was  discovered  that  transmission  was  unreliable  when  both  the 
transmitter  and  receiver  were  turned  on.  In  fact,  it  was  necessary  for  the 
microcontroller  to  turn  off  the  receiver  and  to  wait  a  noticeable  amount 
of  time  for  the  transmitter  to  power  up  before  a  return  packet  could  be 
sent  reliably.  After  much  experimentation  and  frustration,  the  normal 
ack/nack  handshake  that  might  be  expected  to  accompany  such  a 
communication  was  abandoned  during  runtime  operation  because  it  was 
far  too  slow  for  real-time  interactive  performance.  Besides,  there  was 
not  much  the  system  could  do  if  a  particular  packet  was  missed  and  an 
odor  was  not  displayed  on  time.  Elaborate  error  recovery  routines  could 
not  repair  the  past  and  might  jeopardize  the  future.  So,  elaborate  error 
handling  was  abandoned  in  favor  of  status  lights  indicating  valves  being 
activated  and  relying  on  the  user  to  report  that  the  odors  do  not  seem  to 
be  working.  If  a  problem  is  suspected,  the  system  can  be  sent  packets 
which  put  it  into  checkout  mode  during  which  it  does  report  what  it  is 
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doing  back  to  the  PC  at  the  slower  rate  at  which  the  RF  system  can  handle 
bidirectional  communication.  Or,  the  RF  link  can  be  bypassed  by  a  direct 
electrical  connection  for  testing. 

III.E.10.  Odorant  Loading 

In  the  requirements,  the  ability  to  easily  load  and  reload  multiple  odors  in 
a  single  effortless  action  was  cited  as  a  desirable  feature.  That  has  not 
been  achieved.  However,  it  is  easy  to  plug  an  individual  reservoir  into  the 
delivery  device  or  to  unplug  it.  There  are  two  connectors  at  the  top  of  the 
reservoir.  One  plugs  into  the  air  manifold  and  the  other  plugs  into  the 
odorant  delivery  channel.  The  user  pushes  the  connectors  on  the  top  of  the 
reservoir  into  the  fittings  on  the  manifold  and  the  job  is  done  in  a  second. 
The  process  is  no  longer  accompanied  by  dripping  odorant  as  it  once  was, 
which  would  leave  the  investigators  with  powerful  odors  on  their  hands 
that  might  take  days  to  wear  off.  However,  there  is  no  mechanism  for 
simultaneously  loading  multiple  odors  and  loading  the  reservoirs 
themselves  is  a  Somewhat  cumbersome  process. 

To  load  a  reservoir,  one  loads  a  syringe  with  2  cc’s  of  the  liquid  odorant 
from  the  container  in  which  it  is  stored.  Then,  the  syringe  is  fitted  into 
the  end  of  a  tube  which  constitutes  one  end  of  a  fixture  that  has  been 
created  for  this  purpose.  The  tube  carries  the  odorant  through  a  2  micron 
filter  and  then  through  some  more  tubing  which  terminates  in  a  fitting 
that  mates  with  the  input  of  the  reservoir.  Hand  pressure  on  the  syringe 
forces  the  2  cc’s  of  material  into  the  tubing  which  is  used  in  the 
reservoir.  Then,  the  loading  fixture  is  disconnected  and  the  reservoir  is 
either  inserted  into  the  delivery  device  or  plastic  caps  are  placed  over  its 
inlet  and  outlet  fittings  to  prevent  contamination  during  storage.  Then, 
the  loading  fixture  is  flushed  and  back  flushed  with  pure  grain  alcohol. 
This  prefiltering  of  the  odorants  with  a  2  micron  filter  prevents  the  10 
micron  in-line  filters  in  the  delivery  device  from  becoming  clogged 
through  extended  use.  When  2  micron  filters  were  used  in  the  delivery 
device,  they  quickly  became  clogged  and  slowed  down  the  odor  delivery. 

III. E. 11.  Flaws  in  Final  Design 

There  were  several  flaws  in  the  final  delivery  system.  They  have  not  been 
observed  to  cause  a  problem  with  Its  operation  but  nevertheless  raised 
concerns.  Some  of  these  have  been  addressed  and  others  will  await  the 
next  generation  of  design. 
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III.E.H.a.  Bubbles  in  the  Reservoirs 

First,  using  pressurized  air  to  drive  the  odorant  from  the  reservoirs 
through  the  valves  and  onto  the  piezo  crystal  assumed  that  the  air  was 
unable  to  get  past  the  odorant.  If  perfectly  continuous  pressure  were 
maintained  on  the  odorant  that  would  probably  be  what  happened.  However, 
when  the  unit  turned  on,  there  was  an  abrupt  increase  in  pressure  as  the 
pump  brought  the  pressure  on  the  odorant  reservoirs  up  to  10  psi.  Then, 
during  normal  operation,  that  air  pressure  was  monitored  by  a  sensor 
which  tripped  whenever  the  pressure  dropped  below  9.5  psi  causing  the 
the  pump  to  turn  on  until  the  pressure  was  back  at  10  psi.  During  the  time 
that  the  pressure  was  declining  from  10  psi  to  9.5  psi,  the  odorant  could 
move  backwards  in  the  tubing.  Whenever  the  pump  turned  on,  there  was  a 
very  abrupt  increase  in  pressure  resulting  in  mixing  of  the  air  and  liquid, 
which  forced  air  bubbles  into  the  odorant  in  the  reservoir  tubing. 

Fortunately,  the  air  bubbles  did  not  impair  the  functioning  of  the  system. 
When  they  reached  the  valves,  the  pressure  was  so  high  that  they  were 
driven  through  almost  instantaneously.  So,  while  there  was  probably  some 
delay  in  the  onset  of  an  odor  when  a  bubble  is  in  the  odorant  line,  it  was 
not  noticeable  to  the  user.  Nevertheless,  we  were  offended  by  this  and 
sought  to  eliminate  abrupt  changes  in  the  pressure  both  during  startup  and 
during  normal  operation  of  the  feedback  loop.  Now  when  the  device  is 
powered  up,  the  pressure  builds  gradually.  And  when  the  device  is  turned 
off,  the  pressure  bleeds  off  gradually.  During  normal  operation,  we  have 
set  the  sensor  to  detect  a  .1  psi  drop  in  pressure  so  that  the  pump  turns  on 
when  the  pressure  falls  to  9.9  psi.  Because  the  correction  that  needs  to  be 
made  is  less  dramatic,  far  fewer  much  smaller  bubbles  appear  in  the 
odorant  lines. 

The  negative  consequence  of  this  strategy  is  that  the  pump  is  turned  on 
more  often  using  more  power.  Minimizing  power  use  was  the  reason  for 
the  larger  acceptable  range  of  pressure  in  the  first  place.  Using 
proportional  valves  instead  of  solenoid  valves  would  probably  allow  the 
pressure  to  be  controlled  more  smoothly  but  would  result  in  even  greater 
power  use.  In  general,  higher  speed  pressure  regulators  with  much  finer 
control  and  faster  response  times  than  those  we  found  on  the  market 
would  be  desirable.  A  MEMS  device  with  many  parallel  valves  possibly  of 
logarithmically  related  diameters  would  be  ideal  for  this  purpose. 
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III.E.H.b.  Stranded  Odorant 

Another  issue  was  that  the  check  valves  that  prevented  odorant  from 
dripping  out  of  the  delivery  tubing  in  the  manifold  had  an  unexpected  side 
effect.  Something  about  the  internal  geometry  of  the  check  valves 
permitted  the  air  to  get  past  some  of  the  odorant,  stranding  it  there.  Thus, 
while  most  of  the  odorant  was  available  for  display,  a  small  amount  was 
not.  This  was  only  noticed  during  the  flushing  procedure.  After  the 
delivery  tubing  was  flushed  with  pure  alcohol  and  then  with  air,  a  small 
amount  of  liquid  flowed  out  of  the  check  valve  when  the  pressure  was 
removed,  indicating  that  this  liquid  was  not  blown  out  of  the  system 
through  the  valve. 

III.E.11.C.  Priming  Material 

Related  to  this  issue  is  the  fact  that  the  amount  of  odorant  required  to 
prime  the  delivery  channel  is  a  noticeable  fraction  of  the  total  reservoir. 
(The  delivery  channel  consists  of  the  check  valve,  the  tubing  to  the  10 
micron  filter  and  its  fittings,  and  the  flexible  tubing  that  leads  to  the 
rigid  tubing,  and  the  rigid  tubing  itself.)  This  fact  is  not  a  problem  if  the 
odorant  is  all  consumed  in  a  single  use.  The  problem  arises  if  only  a  small 
fraction  of  the  odorant  that  was  originally  loaded  into  the  reservoir  was 
needed  in  a  particular  session.  When  the  reservoir  is  removed  for  storage, 
a  non-trivial  amount  of  material  is  still  left  in  the  delivery  channel  from 
which  it  will  be  flushed  to  keep  it  from  drying  out  and  clogging  the  valves. 
This  material  is  wasted.  The  length  of  the  delivery  channel  is  equivalent 
to  4  inches  of  tubing  which  is  20%  of  the  length  of  the  tubing  in  the 
reservoir. 

For  a  material  that  is  used  only  infrequently  in  any  single  session,  over  a 
number  of  sessions  most  of  it  will  be  flushed  out  of  the  system  rather 
than  being  used  for  odor  display.  During  current  laboratory  operation  with 
easily  acquired  odorant  materials,  wasting  some  of  the  odorant  is  not 
much  of  a  problem.  However  if  custom  odors  are  created,  they  may  include 
very  expensive  raw  materials  as  well  as  being  the  result  of  a  very 
expensive  synthesis  process.  In  this  case,  wasting  a  significant  fraction 
of  the  material  would  be  unacceptable.  To  some  extent,  further 
miniaturization  involving  the  machining  or  micro-machining  of  custom 
valves  and  fittings  will  reduce  the  lengths  and  diameters  of  all  channels 
cutting  down  on  the  amount  of  stranded  material.  In  addition,  it  would 
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help  if  the  valves  could  be  flushed  without  removing  the  odorant  from  the 
system.  In  general,  minimizing  lost  material  is  a  future  design 
requirement. 

MI.E.U.d.  Cannibalized  Components 

Another  issue  that  does  not  affect  the  operation  of  the  system  is  that  the 
piezo  crystal  was  taken  from  a  commercially  available  nebulizer.  Its 
manufacturer  buys  the  crystal  and  drive  circuit  module  from  Japan  and  has 
been  unable  to  get  specifications  on  the  crystal  itself,  a  schematic  of  the 
circuit,  or  sources  for  the  analog  components  in  the  drive  circuit.  Since 
the  nebulizer  costs  only  $15  assembled  and  finding  out  further 
information  seemed  impossible,  it  was  necessary  to  cannibalize  the 
components  from  the  nebulizer  circuit  and  to  place  them  in  the  drive 
circuit  on  the  PC  board  in  the  delivery  device.  While  this  procedure  is 
inelegant  and  labor  intensive,  it  is  almost  certainly  cheaper  to  take 
advantage  of  the  costs  of  a  mass  produced  device  than  to  reverse  engineer 
the  circuit  and  to  try  to  find  equivalent  components  which  would  have  to 
be  purchased  in  small  quantities. 

III.E.12.  Delivery  System  Summary 

The  final  delivery  system  can  be  used  in  the  three  major  virtual  reality 
formats:  desktop,  CAVE,  and  HMD.  It  is  portable,  wearable,  and  wireless,  it 
can  display  one  or  more  of  ten  odors  with  a  50  millisecond  lag  time  which 
is  fast  enough  for  most  immersive  training  applications. 

III.F.  Delivery  System  Deliverable 

The  system  described  above  is  the  deliverable  for  the  project.  The  device 
with  its  own  documentation  is  being  sent  separately  to  the  contracting 
authority.  That  documentation  comprises: 

•  mechanical  design 

•  electronic  design 

•  micro  controller  software 

•  PC  odor  control  and  RF  packet  software 

•  operating  procedures 
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IV.  Wireless  Operation 

The  second  major  area  of  effort  was  in  tracking  the  participant’s  head 
movements  in  an  immersive  virtual  environment. 

This  effort  is  premised  on  the  belief  that  odor  display  and  indeed  visual 
and  auditory  display  in  virtual  environments  would  be  most  effective  if 
these  virtual  sensations  were  encountered  exactly  as  they  would  be  in  the 
real  world.  The  naturalness  standard  implies  unencumbered  behavior  with 
complete  freedom  of  movement.  Current  virtual  reality  technology  offers 
tethered  movement  to  a  significantly  encumbered  participant  wearing  an 
HMD  or  complete  visual  immersion  to  a  stationary  and  much  less 
encumbered  participant  in  a  CAVE.  So  far,  at  least  65  $1M  CAVEs  have 
been  built  which  is  almost  certainly  more  than  the  number  of  $1M  HMD- 
based  immersive  virtual  reality  systems.  This  might  indicate  that  the 
CAVE  is  superior  to  the  HMD,  but  the  comparison  is  not  fair  because  most 
HMD  systems  forego  the  greatest  advantage  that  their  approach  has  to 
offer,  namely  the  ability  to  walk  around  in  the  virtual  world  thereby 
making  use  of  the  lifetime  investment  we  all  have  made  in  interpreting 
the  world  in  terms  of  the  height  of  our  eyes  above  the  ground  and  the 
length  of  our  stride.  Tethered  operation  does  not  answer  the  need,  because 
the  most  important  object  in  the  space — the  wire — is  invisible. 

iV.A.  Wireless  Video  and  Olfactory  Display 

Therefore,  completely  wireless  operation  was  seen  as  the  most  realistic 
way  for  olfactory  stimuli  to  be  encountered  in  immersive  virtual  reality 
scenarios.  The  odor  delivery  system  described  earlier  is  wireless.  The 
virtual  image  is  also  transmitted  wirelessly.  This  was  not  a  matter  of 
simply  using  commercial  wireless  units  for  transmitting  a  video  signal 
from  a  stationary  transmitter  to  a  stationary  receiver.  These  units  are 
susceptible  to  multipath  transmission  problems  which  are  caused  by 
reflections  of  the  signal  from  nearby  objects  leading  to  multiple  copies  of 
the  signal  arriving  at  the  receiver  slightly  displaced  in  time.  In  a 
stationary  setup,  this  problem  can  be  dealt  with  by  adjusting  the 
receiving  antenna.  In  immersive  virtual  reality  where  the  person  is 
wearing  the  antenna  and  is  likely  to  be  turning  often,  these  systems  lose 
synch  constantly  and  often  lose  the  signal  completely.  The  solution  is  a 
diversity  receiver  which  selects  the  best  signal  from  multiple  antennas. 
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IV.B.  Wireless  Tracking 

The  final  component  of  a  completely  wireless  system  is  wireless  head 
tracking.  There  are  two  large-area  tracking  systems  that  have  been 
described  in  the  literature,  the  Hiball  system  at  the  University  of  North 
Carolina  in  which  the  participant  wears  7  position-sensing  diodes  on  his 
head  and  the  Constellation™  system  developed  by  InterSense  Corporation 
which  uses  a  combination  of  inertial  and  ultrasonic  time-of-flight 
measurements.  [Welch][Foxlin]  Both  of  these  systems  require  significant 
instrumentation  of  the  environment;  an  array  of  LEDs  in  the  the  University 
of  North  Carolina  case  and  a  grid  of  ultrasonic  transmitters  in  the 
InterSense  system.  Both  of  these  systems  are  tethered.  InterSense  does 
point  out  the  possibility  of  wearing  a  modem  (suggested  to  them  by  the  PI) 
to  transmit  information  back  to  the  computer  to  achieve  wireless 
operation  but  doing  so  slows  down  the  operation  of  the  system  because 
the  data  must  be  formed  into  packets  and  transmitted  serially. 

Whereas  these  two  systems  are  Inside  out  in  the  sense  that  the 
participant  wears  the  sensors,  the  approach  taken  in  this  project  has  been 
outside  in  with  the  participant  wearing  a  target  which  is  detected  by 
stationary  sensors  positioned  around  him  in  the  environment.  Such  a 
system  is  inherently  suitable  to  wireless  operation  because  the  sensors 
that  determine  the  participants  location  are  mounted  in  the  environment 
where  they  can  be  connected  to  the  computer. 

IV.B.1.  Overhead  Tracking  Approach 

Early  in  the  program,  a  wireless  head  tracking  system  was  developed 
based  on  8  ceiling-mounted  video  cameras  positioned  along  a  40  foot  path. 
This  system  did  provide  1/30  second  x-y  position  and  gaze  direction  (yaw) 
determination.  Pitch  and  roll  had  to  be  averaged  over  several  sample 
periods  to  provide  stable  values.  If  that  system  were  reimplemented  today 
with  the  fast  framing  cameras  that  are  now  available,  it  would  easily  be 
fast  enough  and  accurate  enough  to  compete  with  other  systems.  In  fact, 
the  sensors  that  we  are  using  in  the  final  approach  would  provide  very 
high  speed  data  and  twice  the  accuracy  of  the  original  system.  Recently 
introduced  sensors  would  combine  very  high  speed  with  four  times  the 
resolution  of  the  original  system. 

However,  the  original  approach  had  several  disadvantages.  First,  a  16  foot 
ceiling  was  required  to  allow  the  cameras  to  cover  a  sufficient  area.  This 
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requirement  would  prevent  the  technique  from  being  used  in  many 
laboratory  settings.  Second,  the  cellular  approach  required  the 
installation  of  a  large  number  of  cameras  (64)  to  cover  the  desired 
40’x40’  area.  While  the  cost  of  the  components  would  be  significant,  the 
cost  of  the  installation  itself  might  be  more  so.  Any  system  requiring  a 
significant  installation  is  likely  to  face  significant  resistance.  On  the 
other  hand,  this  system  did  have  the  advantage  that  it  could  handle 
multiple  participants.  The  University  of  North  Carolina  system  which 
involves  dynamic  patterns  in  the  ceiling  LEDs  might  run  into  conflicts  if 
multiple  people  were  present.  The  InterSense  system  would  seem  to  have 
even  more  trouble  with  multiple  participants  because  multiplexing  the 
ultrasonic  channel  would  slow  the  system  down  further. 

IV. B. 2.  Virtual  Camera  Approach 

While  we  felt  that  our  original  approach  had  basically  solved  the  problem, 
we  had  a  second  idea  that  had  a  number  of  advantages  over  that  approach 
as  well  as  those  mentioned  above. 

IV.B.2.a.  Virtual  Camera  Specifications 

This  approach  had  an  array  of  wonderful  advantages: 

•  it  would  allow  coverage  of  a  40  foot  square  area  with  sensors  mounted 

in  the  corners  of  that  space,  so  minimal  installation  would  be  required. 

•  it  would  provide  very  high  speed  operation — over  1000  samples  per 

second.  (10  thousand  samples  per  second  was  possible.)  The  sensor 
circuit  was  built  and  operated  at  speed.  The  software  algorithms  were 
implemented  and  ran  up  to  30  thousands  samples  per  second,  so  the 
hardware  was  the  limiting  factor. 

•  it  would  be  very  accurate — one  millimeter  accuracy  throughout  the 

working  volume. 

•  it  would  work  in  a  typical  office  space  with  an  8  foot  ceiling. 

The  speed  is  important  to  the  accuracy.  All  of  the  analyses  of  tracking 
systems  discuss  accuracy  in  terms  of  a  stationary  target.  None  consider 
the  fact  that  the  target  is  moving.  A  walking  person  is  moving  about  2 
millimeters  per  millisecond  on  the  average.  The  instantaneous  rate  of 
movement  may  be  considerably  higher.  If  the  tracking  technique  depends 
on  sequential  querying  of  multiple  sensors,  then  the  person  may  have 
moved  enough  between  samples  from  the  different  sensors  that  the  values 
from  the  respective  sensors  do  not  constitute  an  instantaneous  sample  in 
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any  sense  at  all.  By  speeding  the  sample  rate,  this  source  of  error  is 
minimized.  In  addition,  a  high  sample  rate  permits  successive  samples  to 
be  averaged  if  there  is  any  instability  in  the  data.  Finally,  the  faster  the 
sampling,  the  more  time  is  available  for  computing  the  system’s  reaction 
to  the  participant’s  behavior  and  generating  the  graphic,  auditory,  and 
olfactory  responses. 

This  approach  involved  creating  a  virtual  camera  with  6000x6000  pixels 
to  be  positioned  in  each  corner  of  the  room.  This  feat  was  accomplished  by 
using  two  line  sensors — one  for  the  horizontal  coordinate  and  the  other  for 
the  vertical.  The  6K  resolution  of  the  line  scan  sensors  explains  the 
resolution  of  the  approach. 

IV.B.2.b.  Virtual  Camera  Optics 

The  trick  is  to  map  the  rectangular  image  produced  by  normal  camera 
optics  onto  each  line  scan  sensor  in  a  way  that  preserves  the  coordinate 
information  in  the  dimension  of  the  line  while  collapsing  all  of  the 
information  in  the  other  coordinate  onto  a  single  pixel.  An  optical  system 
based  on  cylindrical  lenses  was  used  for  this  purpose.  For  this  system  to 
cover  a  square  area  from  the  corners,  it  was  necessary  to  have  the 
equivalent  of  a  wide-angle  lens  in  front  of  the  rectangular  to  linear 
conversion  system. 

IV.B.2.b.i.  Anamorphic  Optics 

In  addition,  it  was  noted  that  while  the  person’s  head  could  be  located 
within  a  90°  field  of  view  horizontally,  it  would  be  confined  to  a  30°  field 
of  view  vertically.  The  result  would  be  that  two  thirds  of  the  resolution 
of  the  line  scan  sensor  would  be  outside  of  this  range  in  a  conventional 
optical  system.  Optical  systems  which  have  an  asymmetric  field  of  view 
are  termed  ana/77orp/7/c.  There  was  no  mention  in  the  literature  of 
anamorphic  fish-eye  lenses.  Further  complicating  the  design  was  the  fact 
that  both  the  line  scan  sensors  and  the  brightest  LEDs  operate  in  the 
infrared  part  of  the  spectrum  which  meant  that  conventional  optics  could 
not  have  been  used  even  if  we  had  been  willing  to  settle  for  symmetric 
optics. 

Dr.  Jannick  Rolland  of  the  Center  for  Research  and  Education  in  Optics  and 
Lasers  (CREOL)  at  the  University  of  Central  Florida  in  Orlando  developed  a 
complete  design  for  this  optical  system.  Unfortunately,  the  cost  of 
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actually  fabricating  these  custom  lenses  turned  out  to  be  $320K  for  the 
eight  optical  systems,  to  which  we  would  need  to  add  the  cost  of  the 
optical  assembly  and  the  printed  circuit  boards  for  the  line  scan  sensors 
which  had  only  been  prototyped  at  that  point.  In  addition,  the  optical 
assembly  was  very  large — almost  2  meters  in  length.  The  cost  was  deemed 
too  high  and  the  size  seemed  to  be  quite  unwieldy.  Therefore,  considerable 
effort  was  expended  in  the  last  year  to  find  design  compromises  that 
would  lower  the  cost  and  decrease  the  size. 

IV.B.2.b.ii.  Optical  Alternatives 

One  of  the  expensive  aspects  of  the  design  was  the  fact  that  the  line  scan 
sensor  had  10  micron  pixels  and  was  therefore  60  mm  long.  The  large  size 
of  the  sensor  necessitated  very  large  lenses  in  the  front  end  of  the  fish 
eye.  Therefore,  we  looked  at  using  a  4K  sensor  from  a  different  vendor 
with  7  micron  pixels  and  therefore  only  28  mm  length. 

We  also  abandoned  the  anamorphic  approach  and  decided  to  live  with  the 
loss  of  vertical  resolution.  To  accommodate  the  reduced  resolution,  we 
reduced  the  working  volume  to  20  feet  square.  Since  the  new  sensor  was 
sensitive  in  the  visible  rather  than  the  IR  region,  we  could  also  use  a 
conventional  fish  eye  lens  design.  Indeed,  we  could  use  an  off-the-shelf 
consumer  product  if  we  could  get  the  complete  optical  characterization  of 
the  device  from  the  manufacturer.  Unfortunately,  none  of  the  lens 
manufacturers  were  willing  to  provide  this  information  which  forced  us 
into  designing  a  custom  fish  eye  that  would  work  with  the  second  stage  of 
the  system.  The  cost  of  the  resulting  system  was  $80K  for  the  optics 
rather  than  the  $40K  that  might  otherwise  have  been  achieved. 

IV.B.2.b.iii.  Non  Optical  Alternatives 

We  then  looked  at  a  variety  of  ways  of  performing  the  rectangular-to- 
linear  conversion  of  the  second  stage  of  the  system  by  non-traditional 
means.  The  PI  tried  getting  a  number  of  firms  to  fabricate  a  two- 
dimensional  version  of  fiber  optics  in  which  planar  elements  would  be 
laminated  together  separated  by  optical  cladding  and  shaped  so  as  to 
reflect  light  entering  from  a  thin  rectangular  cross  section  at  one  end  to  a 
point  on  the  line  scan  sensor  at  the  other. 

He  worked  with  several  sensor  manufacturers  to  get  them  to  fabricate  a 
device  in  which  all  of  the  pixels  in  each  column  were  joined  to  create  a 
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single  elongated  sensor.  While  such  a  device  would  contain  much  more 
silicon,  it  would  eliminate  the  half  of  the  optics  dedicated  to  the  function 

of  collapsing  a  rectangular  image  onto  a  linear  sensor  to  get  the 

horizontal  or  vertical  coordinate  of  the  currently  illuminated  LED  on  the 
tracking  target. 

Dr.  Rolland  developed  a  theoretical  technique  using  a  retroreflective 
screen  as  an  element  in  an  assembly  that  would  perform  the  desired 
transformation.  When  off-the-shelf  screens  were  found  not  to  be  precise 
enough,  3M  was  convinced  to  create  special  retroreflective  screens  whose 
performance  was  closer  to  ideal.  Unfortunately,  these  materials  were 
still  not  good  enough  to  make  the  design  work  in  a  practical  sense. 

Another  approach  was  to  use  an  even  smaller  2K  sensor  and  to  steer  its 

field  of  view  by  means  of  a  mirror  mounted  on  the  shaft  of  a  stepper 

motor.  This  approach  would  apply  the  reduced  resolution  over  a  much 
smaller  field  of  view  to  provide  even  greater  resolution.  Since  the  person 
could  move  out  of  this  narrow  field  of  view,  it  would  be  necessary  to 
move  the  mirror  to  keep  him  in  view.  While  it  would  be  possible  to  make 
measurements  while  the  mirror  was  being  steered,  it  would  be  more 
prudent  for  the  system  to  dynamically  decide  which  pair  of  cameras  had 
the  best  view  of  the  participant  and  to  direct  the  remaining  pair  to  make 
the  mechanical  adjustments  needed  to  improve  theirs. 

IV.B.3.  Fall  Back  System 

While  we  were  trying  to  salvage  the  line  scan  approach,  we  were  looking 
at  a  fall  back  position  based  on  a  new  kind  of  sensor  that  had  recently 
become  available.  One  of  the  frustrations  of  image  processing  has  always 
been  that  one  is  a  prisoner  of  the  raster  scan,  forced  to  read  out  the  entire 
image  even  when  only  interested  in  a  small  fraction  of  it.  There  have 
always  been  analog  video  cameras  with  steerable  scanning,  but  they  were 
too  expensive  for  experimental  use.  However,  the  recent  advent  of  CMOS 
sensors  has  finally  made  such  a  capability  affordable.  These  devices 
permit  the  user  to  read  out  individual  pixels  randomly  with  horizontal 
sequences  being  faster  than  those  that  require  a  new  line  to  be  set.  Given 
that  single  LEDs  are  being  tracked  and  that  these  can  only  move  a  small 
distance  from  one  sample  to  the  next,  it  is  only  necessary  to  scan  the 
window  of  possible  locations  rather  than  the  entire  image.  Thus,  we  can 
track  a  3  or  4  LED  target  at  around  1000  samples  per  second.  It  should  be 
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noted  that  this  speed  can  be  maintained  only  if  the  target  has  been 
initially  captured  and  is  not  lost  during  operation.  If  it  is,  the  participant 
may  be  asked  to  hold  still  for  a  second  while  the  sensors  scan  the  entire 
space  looking  for  the  target. 

IV.B.3.a.  Cameras 

The  cameras  we  purchased  were  only  512x512  resolution  but  the  vendor 
listed  a  2Kx2K  device  based  on  the  same  principles  of  operation  that  they 
said  was  available.  Our  thought  was  to  use  the  inexpensive  device  for 
proving  the  concept  and  then  to  substitute  the  higher  resolution  device 
once  we  were  convinced  that  the  approach  would  work.  With  the  first 
device,  we  felt  that  we  could  cover  a  10  foot  square  CAVE-like  area  with 
better  speed  and  resolution  that  the  tracking  system  currently  used  in  the 
CAVE  which  has  a  very  noticeable  lag. 

Proving  the  concept  turned  out  to  be  difficult  not  because  of  the  problem 
but  because  of  the  state  of  the  products  with  which  we  had  to  solve  it. 
What  had  been  presented  as  off-the-shelf  products  over  time  revealed 
themselves  to  be  evaluation  boards  with  many  unnecessary  quirks  that 
could  have  been  eliminated  with  little  effort  by  the  vendor.  For  instance, 
the  lenses  were  mounted  slightly  off  center  and  at  a  slight  angle  to  the 
camera  board.  When  screwed  all  the  way  into  the  lens  holder,  the  lens  was 
out  of  focus.  It  was  necessary  to  stop  somewhat  short  of  the  end  of  the 
threads.  Each  camera  was  different  from  the  others  necessitating  a 
characterization  process  and  correction  algorithm  unique  to  each  one. 

In  addition,  the  drivers  on  the  camera  boards  would  work  with  only  3  feet 
of  cable  which  was  enough  for  us  to  test  with  a  pair  of  cameras  hooked  up 
to  one  computer  but  which  prevented  us  from  arranging  four  cameras  in 
the  corners  of  the  space  as  required  by  the  design.  Plus,  for  some  reason 
the  cameras  came  with  only  two  device  addresses  which  meant  that  more 
could  not  be  interfaced  to  a  single  computer.  The  fact  that  we  only  had 
two  PCI  slots  in  our  PC  also  forced  to  look  at  more  than  one  computer. 

IV.B.3.b.  Software 

The  software  also  presented  problems.  The  software  that  was  sold  with 
the  cameras  was  written  in  GNU  C  and  was  not  designed  to  run  on 
Microsoft  systems.  Not  all  of  the  code  could  be  accessed.  However,  we 
were  able  to  get  it  running  under  DOS  which  was  acceptable  because  it  is 
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faster  than  Windows  for  low-level  real-time  work.  It  was  necessary  to 
get  32  bit  DOS  Extender  software  to  handle  the  large  arrays  that  were 
needed  by  the  camera  test  software  and  the  desire  to  create  diagnostic 
graphic  software. 

Only  when  we  added  the  third  and  fourth  cameras,  requiring  additional  PCs 
to  control  them,  did  we  discover  that  there  was  no  ethernet  software  with 
sockets  available  on  the  market  that  would  run  under  DOS  operating  in  this 
mode.  Rather  than  trying  to  convert  code  that  was  not  designed  to  operate 
under  Wlndows^"^  to  run  under  that  regime,  we  decided  to  move  the  system 
to  LINUX.  There  were  several  reasons  for  this.  Whereas  Windows™  is 
basically  unknowable  except  through  long  experience,  LINUX  is 
theoretically  transparent  since  its  source  code  is  available.  Also,  we  have 
over  20  years  of  experience  working  with  UNIX  systems.  Finally,  it  is 
likely  that  a  real-time  version  of  LINUX  will  be  developed  which  strips 
away  all  the  overhead  of  being  ready  to  handle  every  possible  option  that 
any  of  100  million  users  in  the  world  might  want  at  any  given  moment. 
What  is  desirable  is  an  old-fashioned  Load-and-Go  operating  system  which 
provides  services  during  the  programming  process  but  which  disappears 
when  the  real-time  application  takes  over  the  machine  and  only  uses  those 
operating  system  functions  that  the  application  requires. 

IV.B.3,c.  Multiprocessing 

Given  the  problem  of  installing  more  than  two  cameras  in  a  single 
computer  and  the  3  foot  limitation  on  the  cabling,  we  resolved  to  dedicate 
a  PC  to  controlling  each  camera,  to  correcting  for  the  idiosyncrasies  of 
each  camera  and  for  the  distortions  of  the  wide  angle  lenses,  and  to 
computing  the  rays  along  which  the  target  LEDs  were  found.  A  fifth 
computer  then  integrates  the  information  from  the  four  cameras  to 
determine  the  location  and  orientation  of  the  target  in  three  dimensions. 
This  machine  also  controls  the  smart  target.  A  sixth  computer  displays 
diagnostic  information  in  graphic  form. 

Having  been  forced  into  a  multiprocessor  system  and  into  learning  LINUX 
has  taken  an  unexpected  investment  in  time  but  has  the  advantage  in  that 
it  is  a  competent  system  and  will  not  be  subject  to  the  capricious 
upgrades  that  jeopardize  every  effort  done  on  PC  operating  systems.  On 
the  other  hand,  we  have  still  not  found  any  way  to  initialize  the  cameras 
under  LINUX  so  the  system  is  brought  up  under  DOS  first. 


The  major  hurdle  to  implementing  this  computer-as-a-component  system 
was  getting  the  ethernet  to  operate  in  the  consistent  way  that  we  needed 
for  our  real  time  application.  Under  its  default  settings,  the  ethernet  is 
treated  like  a  resource  to  be  utilized.  The  goal  is  maximum  total  traffic, 
not  rapid  response.  Messages  sent  one  moment  are  not  guaranteed  to  be 
received  at  the  destination  the  next.  Instead,  they  may  be  queued  for  some 
time  on  the  originating  computer  before  they  are  sent  or  may  not  be 
reported  to  the  application  that  is  to  use  the  data  the  moment  they  are 
received. 

This  default  is  not  consistent  with  the  needs  of  a  real-time  application 
operating  across  an  ethernet  that  it  dedicated  to  it.  In  this  case,  what  is 
required  is  consistent  behavior  characterized  by  instantaneous  response. 
The  lower  levels  of  the  ethernet  protocols  contain  mechanisms  for 
enabling  such  communications.  Messages  are  not  sent  from  computer  to 
computer  so  much  as  from  part  of  a  process  running  on  one  computer  to 
another  part  of  that  process  running  on  another.  The  sockets  that  enable 
such  communication  also  speed  up  the  transmission  by  eliminating  much 
of  the  overhead  that  is  unnecessary  in  such  a  dedicated  network. 

The  resulting  communication  over  a  100  MB  ethernet  line  is  fast  enough 
not  to  represent  a  bottleneck  in  the  tracking  application,  and  the  entire 
process  of  camera  handling,  interprocessor  communication,  and  data 
integration  takes  place  faster  than  500  times  a  second. 

IV.B.S.d.  Status 

The  current  status  is  that  this  third  wireless  tracking  system  does 
control  the  tracking  of  head-worn  LED  targets  by  four  cameras  and 
integrates  the  results  to  produce  3D  coordinates  and  orientations.  It  does 
so  500  times  a  second  and  produces  stable  output  values.  However,  it 
currently  handles  only  benign  orientations.  The  hand  off  from  pairs  of 
cameras  to  select  the  pair  with  the  best  view  of  the  target  has  not  been 
done  nor  has  the  smart  target  function  been  implemented.  Nor  has  head 
movement  modelling  been  done.  This  is  not  necessary,  but  it  would  allow 
us  to  decrease  the  size  of  the  windows  that  we  are  currently  scanning  and 
to  increase  the  sample  rate  even  further.  Since  the  current  sample  rate  is 
much  faster  than  the  graphic  generation  rate,  this  is  not  a  priority. 
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IV.B.3.d.i.  Smart  Targets 

Currently,  the  system  operates  with  three  or  four  targets  facing  opposite 
directions,  each  consisting  of  three  LEDs.  These  targets  are  easily 
viewable  from  all  directions  as  long  as  the  head  is  tilted  no  more  than  45° 
in  any  direction.  Greater  tilting  can  probably  be  tolerated,  but  possibly  not 
in  all  locations  in  the  environment.  To  accommodate  a  greater  range  of 
angles  and  positions,  we  will  implement  a  smart  targeting  system.  With 
this  approach,  a  larger  number  of  LEDs  will  be  distributed  around  the  head 
and  the  system  will  dynamically  select  a  subset  of  them  to  create  a 
target  which  can  be  easily  viewed  by  at  least  one  pair  of  cameras.  With 
this  technique  the  system  will  be  able  to  handle  the  problems  that  arise 
when  the  LEDs  on  a  fixed  target  would  have  to  be  viewed  at  very  oblique 
angles.  The  detection  of  the  LEDs  is  difficult  at  such  angles  and  the 
possibility  that  two  LEDs  will  line  up  merging  into  a  single  spot  of  light 

becomes  real.  The  smart  targets  will  permit  the  person’s  head  to  tilt 

much  further  from  the  vertical,.  It  will  not  permit  them  to  lie  down 

wherever  they  want  facing  in  any  direction.  However,  a  medic  should  be 

able  to  bend  over  a  casualty  and  an  infantryman  will  be  able  to  lie  down  in 
the  center  of  the  working  volume. 

IV.B.S.d.ii.  Recapture  of  Lost  Target 

We  have  also  not  dealt  with  the  issue  of  what  to  do  when  the  tracking 
system  loses  lock  on  the  target.  Currently,  the  person  stands  still  for  a 
moment  while  the  system  finds  him.  In  the  small  working  volume  the 
current  system  can  handle,  we  do  not  expect  that  the  user  will  outrun  the 
system.  However,  it  is  possible  that  they  will  raise  their  hand  or  even 
both  hands,  momentarily  occluding  the  targets  from  the  view  of  any 
sensor  pairs.  In  this  case,  the  system  will  face  the  more  daunting  task  of 
trying  to  find  a  person  as  they  are  moving  around. 

This  requires  searching  a  larger  window  which  will  slow  the  system  down 
considerably.  The  brute  force  solution  is  to  have  every  camera  scan  its 
entire  field  of  view  which  can  be  accomplished  at  no  more  than  3  samples 
per  second.  While  it  might  seem  that  capture  would  be  guaranteed  within 
1/3  second,  this  is  not  so.  At  this  glacial  scanning  rate,  it  is  quite 
possible  for  a  person  to  outrun  the  scan  so  they  remain  invisible  for  a 
considerably  longer  period  of  time.  A  perverse  individual  could  thwart 
recapture  indefinitely.  A  more  likely  scenario  is  that  one  of  the  cameras 
will  detect  at  least  one  of  the  LEDs  on  the  first  scan.  The  location  of  that 
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LED  in  the  scan  will  constrain  where  the  person  can  be  enough  to  shrink 
the  search  window  on  all  cameras  on  the  next  scan.  Each  scan  will  improve 
the  information  so  that  by  the  end  of  a  second  or  two,  the  system  will  be 
locked  onto  the  participant.  Since  all  of  the  cameras  will  see  a  given 
target  at  the  same  vertical  elevation,  they  can  each  be  given  a  vertical 
quadrant  to  search,  cutting  the  capture  time  by  a  factor  of  four. 

However,  what  is  the  graphics  doing  during  this  time  and  what  is  the 
participant  doing?  In  the  absence  of  up-to-date  tracking  information,  the 
system  can  decide  to  use  a  predicted  location  for  the  graphics  or  it  can 
simply  freeze  the  graphics  on  the  most  recent  correctly  computed  frame. 
When  the  participant  experiences  the  loss  of  optical  flow,  he  is  unlikely 
to  continue  moving  as  if  nothing  has  happened.  Instead,  he  is  likefy  to 
pause,  possibly  for  enough  time  for  the  recapture  to  take  place. 

IV.B.3.d.iii.  Accuracy 

We  have  not  measured  the  absolute  accuracy  of  the  system  and  are  not  as 
concerned  about  that  as  with  the  relative  accuracy  and  monotonicity.  At 
any  moment,  an  HMD  wearer  thinks  he  is  where  his  eyes  tell  him  he  is  in 
the  virtual  world.  It  is  not  important  where  he  actually  is  in  the  real 
environment  unless  he  is  about  to  bump  into  something  in  the  real 
environment.  What  is  most  important  is  that  the  image  does  not  jitter.  Of 
next  importance  is  that  if  he  moves  his  head,  the  resulting  changes  in 
what  he  sees  are  the  ones  that  he  expects. 

Absolute  accuracy  is  most  important  in  augmented  reality  system  in 
which  the  user  can  compare  virtual  objects  with  physical  objects  and  is 
quite  offended  by  obvious  discrepancies  between  their  positions.  The 
current  system  would  provide  such  accuracy  only  in  a  small  working 
volume  as  might  be  involved  in  an  assembly  task. 

The  original  plan  was  to  substitute  2Kx2K  cameras  for  the  512x512 
cameras  as  soon  as  the  system  was  working  properly  with  the  lower 
resolution  devices.  With  the  high-resolution  cameras,  the  tracking  system 
would  have  provided  satisfactory  accuracy  at  high  sample  rates  over  a 
20’x20’  area.  This  would  have  been  a  large  enough  area  to  support  many 
useful  simulations  of  emergency  medical  scenarios  for  dismounted 
personnel.  Unfortunately,  the  vendor  removed  the  2Kx2K  devices  from  the 
market  because  it  felt  that  they  were  not  living  up  to  the  performance 
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specifications  that  the  company  had  claimed  for  them.  Very  fast  framing 
1Kx1K  sensors  are  now  available  and  offer  another  alternative. 
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V.  Odor  Representation 


Representing  the  odors  needed  for  virtual  reality  simulations  turned  out 
to  be  the  greatest  barrier  to  using  odors  in  virtual  reality  training 
simulations,  especially  for  medical  training  applications.  While  our  initial 
collaborators  at  Monell  Chemical  Senses  Center  assured  us  that  getting 
many  of  the  odors  we  would  need  would  be  no  problem,  it  turned  out  that 
getting  any  of  the  appropriate  odors  was  a  challenge.  Of  the  odors  that 
could  be  acquired,  very  few  were  dead-on  duplicates  for  the  real  thing. 

Most  of  the  medical  odors  that  Monell  was  able  to  produce  were 

completely  unconvincing.  This  was  also  the  case  for  the  two  odors  that 
were  provided  by  International  Flavors  and  Fragrances  Corporation.  Only 
Monell’s  sweat  smell  could  be  judged  as  a  complete  success.  However, 
that  formulation  was  not  developed  under  this  project.  Instead,  it  was  the 
single  result  of  a  multiyear  research  project  itself  costing  hundreds  of 
thousands  of  dollars. 

V.A.  Fragrance  Industry  Practice 

On  the  one  hand,  the  odor  duplication  process  is  very  sophisticated  and  on 
the  other  it  seems  very  primitive.  The  world’s  200  perfumers— each 
making  $200K  per  year— are  all  associated  with  large  corporations.  Those 
who  purport  to  be  free  lance  agents  need  the  facilities  of  the  big  firms  to 

ply  their  trade.  Access  to  those  facilities  requires  a  clear  business 

opportunity  representing  tens  of  millions  of  dollars  in  sales  in  which  the 
large  firm  expects  to  participate. 

The  facilities  of  a  fragrance  lab  include  elaborate  analytical  equipment 
supported  by  PhD  chemists  and  laboratory  technicians.  The  equipment 
typically  includes  a  gas  chromatograph  connected  to  a  mass  spectrometer. 
The  gas  chromatograph  separates  the  ingredients  of  a  vapor  sample  and 
the  mass  spectrometer  breaks  its  molecules  into  fragments  which  can  be 
identified  and  from  which  the  original  complex  molecules  can  be  inferred. 
Since  any  naturally  occurring  odorous  compound  may  contain  hundreds  of 
components  many  of  which  do  not  contribute  to  the  odor  at  all,  fragrance 
labs  have  what  is  called  a  sniffer  port  between  the  gas  chromatograph  and 
the  mass  spectrometer  which  the  perfumer  can  use  to  smell  the 
constituents  that  are  output  from  the  chromatograph  to  see  if  they  are 
important  to  the  overall  odor. 
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However,  the  human  nose  is  more  sensitive  to  a  wider  range  of  odors  than 
any  instrument  and  there  are  likely  to  be  components  that  are  observed 
that  cannot  be  detected  or  identified  experimentally.  In  this  case,  the 
perfumer  may  be  able  to  identify  it  by  smell  from  the  sniffer  port  and  it 
may  be  possible  to  substitute  a  material  with  a  similar  scent.  Since  there 
are  many  different  types  of  instruments  each  with  different  strengths  and 
weaknesses,  the  PI  is  not  sure  whether  it  is  simply  the  case  that  no  single 
instrument  has  the  sensitivity  of  the  human  nose  or  whether  there  odors 
for  which  no  instrument  can  duplicate  human  performance. 

On  the  one  hand,  duplicating  real  world  smells  is  not  a  priority  of  the 
fragrance  industry.  On  the  other,  analyzing  and  duplicating  the  smell  of 
competitors’  products  is  probably  its  primary  activity.  It  resembles  the 
fashion  industry  in  the  sense  that  new  fragrance  ideas  appear  in  high-end 
perfumes  which  are  then  imitated  at  lower  cost  by  competitors  and  are  in 
turn  imitated  by  other  competitors  in  other  categories  until  detergent 
manufacturers  may  be  influenced  to  add  aspects  of  the  scent  to  their 
products. 

V.B.  State  of  Odor  Synthesis 

However,  there  are  a  number  of  biases  operating  in  the  fragrance  industry 
which  is  in  the  business  of  pleasing  people.  Therefore,  they  have  almost  no 
interest  in  unpleasant  odors.  There  are  two  exceptions.  First,  some 
unpleasant  ingredients  can  contribute  to  a  pleasant  result  when  used  in 
small  quantities.  Second,  manufacturers  of  products  designed  to  eliminate 
or  at  least  mask  unpleasant  odors  may  develop  synthetic  versions  of  the 
unpleasant  odor  material  they  are  working  against.  For  example, 
corporations  have  developed  analogs  of  sweat,  baby  vomit,  human  urine, 
cat  urine,  feces,  etc.  The  PI  has  talked  to  individuals  in  the  industry  that 
have  spent  their  entire  careers  imitating  just  one  of  those  odors. 
However,  these  formulations  are  closely  guarded  trade  secrets.  They  are 
not  available  for  virtual  reality  research. 

If  you  look  at  the  odors  that  have  been  developed,  there  is  an  emphasis  on 
fruits  and  flowers  which  by  and  large  have  pleasant  odors.  Vegetable 
smells  are  much  less  developed,  although  there  are  many  ingredients 
described  as  herbaceous  or  grassy.  It  is  likely  that  food  flavorists  work 
with  these  odors,  but  they  create  materials  that  are  ingested  not  smelled 
through  the  air.  The  PI  serves  the  Fragrance  Foundation  on  the  jury  for  the 
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technological  innovation  of  the  year  award.  At  the  1998  meeting,  Takasago 
had  several  very  good  examples  of  smells  for  organically  grown 
vegetables  such  as  tomatoes  and  cucumbers. 

Meats  and  therefore  human  tissues  are  much  less  studied.  The  fine 
fragrance  industry  would  have  no  interest,  but  one  would  think  that 
manufacturers  of  canned  foods  and  frozen  meals  would.  Many  of  the  odors 
associated  with  medical  procedures  come  from  cleaners  and  antiseptics. 
Some  of  these  such  as  alcohol  can  be  used  directly.  Others  have 
ingredients  that  have  been  added  to  give  them  a  smell.  For  instance, 
anesthesia  is  sometimes  given  an  odor  even  though  it  typically  has  no 
smell  of  its  own. 

Man-made  materials  and  the  odors  associated  with  emergency  medical 
scenarios  have  a  number  of  additional  problems.  First,  existing  fragrance 
customers  have  no  interest  in  such  odors.  Second,  they  are  terrified  of  the 
ingredients  and  do  not  want  the  liability  of  handling  them  or  of  foisting 
them  on  the  public.  Third,  odors  such  as  gasoline,  diesel  fuel,  and  motor 
oil  as  well  as  their  combustion  products  often  include  particles  and 
droplets  as  well  as  vapors  which  adds  a  completely  new  issue  to  the 
problem  of  duplicating  them.  Some  of  these  ingredients  are  changing 
chemically  as  they  are  being  smelled  and  so  would  be  impossible  to 
duplicate  with  stable  ingredients.  Smoky  smells  exist  for  wood  and  meat 
but  burning  fuels  and  electrical  insulation  are  not  available.  Explosives 
have  not  been  touched. 

Even  if  the  complete  analysis  of  every  ingredient  in  a  real  world  smell 
was  possible  with  current  instrumentation,  a  perfect  simulation  is  often 
not  possible.  Some  of  the  ingredients  may  never  have  been  synthesized. 
Some  may  not  be  available  commercially  or  may  be  too  expensive.  In  no 
case  is  duplicating  the  hundreds  or  thousands  of  ingredients  in  a  natural 
odor  considered  practical.  Instead,  it  is  a  matter  of  identifying  those 
which  are  most  important  in  giving  the  odor  its  characteristic  bouquet.  In 
many  cases  of  common  odors  like  coffee,  the  imitation  is  a  pale 
representation  of  the  real  thing. 

V.C.  Odor  Experience 

Compounding  the  problem  is  that  we  do  not  know  very  much  about  everyday 
odor  experience.  On  the  one  hand,  we  know  that  humans  are  capable  of 
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extremely  sophisticated  discriminations  and  on  the  other  we  know  very 
little  about  what  people  smell  during  the  day  and  whether  they  notice  it 
and  if  they  do  not  are  they  nevertheless  subconsciously  affected  by  it. 
Only  a  few  studies  have  been  done  in  which  people  record  what  they  smell. 
In  at  least  one  case,  an  alarm  has  been  used  to  remind  the  subject  to 
record  what  they  were  smelling  at  that  moment.  It  seems  that  smelling  is 
a  little  like  dreaming.  Everybody  does  it,  but  most  people  do  not  remember 
doing  it  most  of  the  time. 

This  ignorance  may  imply  that  odors  are  not  very  important  to  20th 
Century  Man.  Consciously,  this  may  be  true.  On  the  other  hand,  there  is 
evidence  that  subconscious  perception  of  odors  can  have  significant 
effects  on  our  behavior.  The  most  familiar  example  of  this  is  the  fact  that 
women  living  together  tend  to  synchronize  their  menstrual  periods.  In 
addition,  one  odor  researcher  who  is  also  a  marriage  counselor  reports 
that  if  one  of  the  partners  does  not  like  the  smell  of  the  other,  the  union 
cannot  be  saved.  Do  such  distastes  operate  among  men  who  must  work 
together  in  military  units?  Do  differences  in  diet  lead  to  differences  in 
body  odor  that  cause  friction  in  multinational  peace  keeping  forces?  Are 
there  other  profound  influences  that  remain  to  be  discovered? 

Similarly,  we  know  that  people  have  very  great  differences  in  their 
abilities  to  detect  odors.  Two  orders  of  magnitude  difference  in 
sensitivity  is  common.  In  addition,  specific  anosmias — blindnesses  to 
particular  odors — are  common.  However,  the  only  large  population  study 
that  has  been  done  was  in  the  pages  of  National  Geographic.  We  know  that 
there  are  a  thousand  genes  that  code  for  the  olfactory  receptors  and 
assume  that  there  are  as  many  different  kinds  of  receptor 
cells. [Glausiusz]  We  further  know  that  as  many  as  30%  of  them  appear  to 
be  too  damaged  to  function.  We  do  not  yet  know  what  chemicals  or 
chemical  characteristics  these  genes  code  for.  We  do  not  know  whether 
there  is  a  universal  set  of  1000  or  whether  there  is  a  much  larger  pool  of 
which  we  each  get  no  more  than  a  thousand. 

We  also  do  not  know  how  odors  are  encountered  in  the  real  world.  All  of 
our  efforts  to  display  odors  are  based  on  the  certainly  misguided 
assumption  that  odors  occur  in  neat  gradients  emanating  from  their 
sources.  It  is  almost  certainly  the  case  that  odors  are  mixed  non 
uniformly  in  the  real  world  and  that  instead  of  a  consistent  odor  intensity 
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we  experience  rapidly  changing  sensations  as  we  move  in  and  out  of  odor 
plumes  and  are  buffeted  by  complex  air  flows.  Evidence  for  this  would 
include  the  fact  that  one  of  the  popular  heroes  of  the  olfactory  world,  the 
moth,  cannot  follow  gradients  as  has  long  been  assumed. [Taylor]  Instead, 
moths  fly  into  an  odor  plume  and  turn  upwind.  When  they  fly  out  of  it,  they 
reverse  course  and  either  fly  back  into  it  or  they  wander  around  randomly 
until  they  do.  Placed  in  a  carefully  constructed  gradient,  they  are  lost. 
This  behavior  suggests  not  only  that  moths  cannot  follow  gradients  but 
also  that  gradients  may  not  be  an  important  aspect  of  odors  in  the  real 
world.  If  they  were,  you  would  expect  moths  to  follow  them  much  as 
lobsters  do  in  the  sea. 

Not  only  are  we  likely  to  encounter  ever  changing  concentrations  of  any 
given  odor,  but  since  these  odors  have  many  different  components  each 
with  different  physical  properties,  the  proportions  of  the  ingredients  that 
we  associate  with  a  given  smell  is  likewise  changing  moment  by  moment. 
For  sure,  odors  coming  from  any  given  source  are  likely  to  change  through 
time  as  the  more  volatile  components  evaporate,  as  heavier  components 
sink  to  the  ground,  and  as  bacteria  act  on  what  remains  to  produce  new 
ingredients.  Sweat  certainly  changes  in  this  way.  Thus,  what  we  have  been 
treating  as  single  odors  composed  of  fixed  ratios  of  components  are 
almost  certainly  more  complex  than  that.  We  do  not  know  how  smelling 
interacts  with  physical  behavior.  (We  do  not  even  know  how  vision 
interacts  with  physical  behavior.)  We  know  nothing  of  these  issues.  No  one 
does. 


V.D.  Past  odor  efforts 

The  original  efforts  at  odor  representation  were  undertaken  by  Monell.  IFF 
also  contributed  a  few  candidate  odors.  A  number  of  other  fragrance  firms 
were  approached  and  a  professional  perfumer  worked  with  us  for  a  while 
hoping  to  get  one  of  the  big  firms  to  support  this  part  of  our  project.  After 
considerable  effort,  we  were  unable  to  garner  any  interest  from  the 
industry.  We  also  approached  universities  that  might  have  some  of  the 
capabilities  including  the  Flavors  Laboratory  at  Cornell.  To  our  surprise, 
they  had  little  interest. 

Therefore,  we  attempted  to  get  the  odorant  chemicals  for  ourselves  and  to 
see  if  any  of  the  raw  chemicals  could  be  made  to  serve  our  purposes.  The 
chemical  firms  did  not  want  to  sell  directly  to  us  because  we  were  not  a 


48 


chemistry  laboratory  and  universities  did  not  want  to  ask  their  students 
to  handle  the  raw  materials  which  are  toxic  in  neat  form  which  is  how 
they  are  sold.  After  many  attempts,  we  were  able  to  get  a  local  lab  to 
purchase  the  chemicals  and  to  dilute  them  down  to  a  concentration  that 
was  safe  to  handle.  This  solution  backfired  when  we  discovered  that  the 
concentrations  while  seemingly  correct  for  smelling  in  the  bottle  were 
too  weak  when  displayed  through  our  delivery  device. 

This  convinced  us  of  the  need  to  make  the  dilutions  with  respect  to  the 
odor  delivery  system  itself.  Since  the  laboratory  we  were  dealing  with 
charged  $175  per  hour  for  its  services,  it  was  not  practical  to  take  the 
delivery  device  to  their  premises  to  work  out  the  concentrations. 

Instead,  we  found  that  we  could  get  the  chemical  suppliers  to  ship  us  one 
or  two  chemicals  at  a  time  without  complaint.  We  diluted  these  materials 
in  a  very  well  ventilated  out  building  while  wearing  special  clothing, 
gloves,  and  a  mask.  In  this  way,  we  were  able  to  test  approximately  50 
chemicals  with  the  delivery  system. 

[Begin  proprietary 
V.E.  Odor  Mixing 

While  only  a  few  of  these  were  ready  for  prime  time  by  themselves,  the 
process  of  readying  them  for  use  was  worked  out  and  we  were  able  to 
begin  what  would  be  the  next  step,  the  use  of  the  odor  delivery  system 
itself  to  mix  together  composite  odors  from  individual  chemicals.  Given 
that  most  odors  are  complex  mixtures  of  many  ingredients  and  that  in 
almost  no  cases  are  all  of  the  ingredients  represented  in  a  simulated  odor, 
mixtures  of  the  10  most  prominent  ingredients  would  be  the  practical 
limit  in  many  cases.  In  cases,  where  the  10  most  prominent  ingredients 
are  known,  the  delivery  system  could  be  used  to  vary  the  proportions  of 
these  under  the  control  of  a  trained  perfumer. 

Why  would  professionals  be  interested  in  mixing  chemicals  with  our 
delivery  device  when  they  have  expressed  no  interest  in  coming  up  with 
the  same  formulations  using  their  traditional  techniques?  The  answer  lies 
in  the  way  that  odor  mixtures  are  formulated.  The  perfumer  sits  in  a 
laboratory  surrounded  by  5000  odorant  chemicals  all  in  liquid  form.  From 
this  inventory  she  has  chosen  a  small  subset  of  from  10  to  50  ingredients 
which  she  is  planning  to  work  with.  Then,  she  adds  one  of  the  ingredients 
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to  a  beaker,  then  another  and  so  on.  After  observing  this  process  for  a  few 
minutes,  I  was  struck  by  the  fact  that  the  perfumer  could  not  back  up.  If 
she  had  too  much  of  one  ingredient,  it  was  almost  impossible  to  reduce  its 
contribution  without  tediously  computing  new  amounts  for  each  of  the 
other  ingredients.  If  she  decided  that  the  new  ingredient  did  not  work  at 
all,  she  was  out  of  luck.  She  had  to  start  over. 

While  the  perfumer  typically  starts  with  a  good  idea  of  the  kind  of  scent 
she  is  looking  to  create,  the  process  is  laborious  and  costly  because  the 
ingredients  that  the  perfumer  works  with  are  often  expensive 
formulations  in  their  own  right. 

It  would  seem  infinitely  easier  not  to  work  with  liquids  at  all.  Instead, 
one  could  mix  vapors.  If  the  vapors  can  be  mixed  in  the  proper  proportions, 
the  perfumer  has  no  investment  in  those  proportions.  Only  a  tiny  amount  of 
material  is  involved  at  any  moment.  And  the  effort  required  to  achieve 
those  proportions  is  infinitesimal.  Proportions  can  be  changed  in  seconds. 
The  experimentation  with  odor  mixtures  can  be  speeded  up  by  orders  of 
magnitude,  saving  money  and  more  importantly  time. 

While  the  perfuming  process  is  undoubtably  much  more  complex  than  it 
was  just  described  to  be  and  the  10  odor  delivery  system  that  the  PI  has 
developed  may  not  handle  enough  chemicals  or  offer  sufficiently  fine 
control,  it  is  possible  that  the  ability  to  speed  up  exploratory 
experimentation  would  change  the  fragrance  development  process  itself. 

For  instance,  consider  the  genetic  algorithms  developed  by  Karl  Sims  when 
he  was  at  Thinking  Machines. [Sims]  He  created  insanely  complex  equations 
in  two  variables  to  assign  color  values  to  each  pixel  in  a  two-dimensional 
image.  By  randomly  varying  the  coefficients  of  these  equations,  he 
produced  four  variations  on  the  first  image.  The  user  selected  the  one  that 
pleased  him  the  most.  Then,  four  variations  on  the  new  image  were 
created  and  the  process  continued  until  the  user  was  satisfied  with  the 
final  image  that  resulted  from  repeated  applications  of  the  basic  process. 
The  interesting  thing  is  that  often  a  user  with  no  artistic  training  or 
talent  was  able  to  produce  a  very  pleasing  image  that  he  would  have  been 
incapable  of  generating  with  conventional  artistic  tools.  Applying  the 
same  idea  to  fragrance  development  suggests  that  if  the  most  salient 
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components  of  a  particular  smell  were  loaded  into  the  delivery  system,  a 
non-perfumer  might  be  able  to  do  as  well  as  the  professional.  (Remember 
that  the  professionally  produced  materials  were  often  not  convincing.) 

While  we  did  not  implement  genetic  algorithms  for  this  purpose,  we  did 
try  the  idea  with  basic  pine  smells  which  are  made  up  of  three  primary 
ingredients.  We  were  able  to  acquire  two  of  the  three  ingredients  and  to 
vary  their  proportions  at  will  with  the  delivery  system.  While  it  is 
impossible  to  claim  that  we  were  producing  perfect  simulations  of  any 
given  variety  of  pine  tree,  it  is  possible  that  we  were  and  just  did  not 
know  it.  What  we  were  doing  was  producing  a  variety  of  evergreen-like 
smells  that  could  be  used  to  provide  texture  to  an  outdoor  scene  which 
contained  a  variety  of  unidentifiable  evergreen  plants, 
end  PROPRIETARY 

V.F.  Impressionistic  Odors 

This  raises  a  question  about  odor  display  in  general.  Is  the  goal  to 
perfectly  represent  the  odors  that  are  present  in  particular  real  world 
situations?  Given  that  the  state  of  our  knowledge  of  our  own  odor 
experience  and  the  fact  that  that  the  modern  world  almost  never  requires 
us  to  identify  unknown  odors  or  to  correctly  track  natural  odors  to  their 
sources,  it  is  not  unreasonable  to  ask  whether  the  goal  is  to  represent  a 
particular  reality  that  smells  in  a  very  exact  way  or  whether  it  is  simply 
to  create  a  simulated  world  that  smells  believable,  even  though  no  real 
world  smells  exactly  like  it.  Such  an  impressionistic  approach  to  odor 
representation  is  probably  necessary  if  we  are  to  avoid  the  extreme 
expense  of  proceeding  the  way  the  fragrance  industry  traditionally  does. 

By  extension,  such  an  approach  may  be  necessary  if  we  are  to  represent 
the  odors  of  hazardous  materials.  For  instance  while  diesel  fuel  is  a 
common  smell,  a  number  of  its  components  including  benzene  are 

hazardous.  When  the  ERA  had  an  interest  in  determining  the  effects  of 
exposure  to  the  smell  of  diesel  fuel  as  opposed  to  its  hazardous 

ingredients.  Dr.  Amos  Turk  created  a  formulation  which  duplicated  the 
qualities  of  diesel  fuel  without  using  those  ingredients. [Turk]  His  mixture 
contained  chemicals  which  stood  for  the  perceived  qualities  of  this 

material,  i.e.  burnt,  oily,  pungent,  aldehydic.  Although  the  resulting 
mixture  did  not  smell  like  diesel  fuel,  it  was  deemed  successful  in 
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determining  how  an  odor  like  diesel  fuel  would  affect  people’s 
performance.  While  odor  duplication  was  explicitly  not  the  goal  of  Turk’s 
work,  it  is  likely  that  a  similar  approach  could  be  taken  to  odor 
representation.  Indeed,  it  is  almost  certainly  part  of  the  approach  that 
professional  perfumers  employ  when  they  try  to  create  a  knock  off  of  an 
expensive  fragrance  with  inexpensive  ingredients. 

For  odors  to  be  an  important  part  of  virtual  reality,  it  is  not  enough  to 
simply  take  readily  available  odorants  and  to  present  them  at  the 
appropriate  moments  in  simulations.  A  library  of  odors  that  smell 
convincingly  real  needs  to  be  gathered.  We  thought  that  we  would  be  able 
to  do  that,  but  found  that  we  were  unable  to  get  the  interest  of  those  with 
the  skills  and  unable  to  buy  their  services  with  mere  money.  Ironically,  we 
are  confident  that  there  is  a  commercial  opportunity  for  the  fragrance 
house  that  takes  on  this  challenge.  Indeed,  the  fragrance  industry  is 
already  plagued  with  mature  markets  and  overcapacity.  Unless  they 
broaden  their  definition  of  the  business  that  they  are  in,  they  all 
recognize  that  there  will  be  a  shakeout. 

V.G.  Hazardous  Materials 

A  final  concern  about  odor  representation  is  that  it  is  greatly  complicated 
by  the  fact  that  many  of  the  materials  are  listed  as  hazardous  in  neat 
form.  However,  there  is  little  indication  in  these  listings  of  the  degree  of 
hazard  or  the  level  of  exposure  for  which  there  is  no  hazard.  The  mere 
possibility  that  circumstances  exist  under  which  a  material  can  present  a 
hazard  is  not  very  enlightening.  89  people  drowned  in  molasses  in 
downtown  Boston  a  hundred  years  ago.  Would  listing  it  as  a  hazardous 
substance  be  appropriate?  Many  odorant  materials  are  clearly  dangerous. 
Some  are  sensitizers  in  that  the  effects  of  successive  exposures  are  more 
severe.  However,  many  everyday  substances  to  which  people  are  routinely 
exposed  such  as  diesel  fuel  and  diesel  exhaust  fumes  contain  toxic 
materials.  It  would  be  nice  to  be  able  to  distinguish  materials  which  are 
instantly  lethal  from  those  which  require  long  term  exposure  in  high 
concentration  to  cause  harm. 

In  general,  the  issue  of  odor  design  and  odor  representation  are  perceived 
to  be  of  small  significance  to  the  larger  community  and  of  interest  to 
those  in  the  business  only  in  so  far  as  there  is  an  immediate  payoff. 
Recent  research  suggests  that  particular  odorants  have  real  effects  on 
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performance.  However,  these  results  are  based  on  short  term  studies.  The 
possibility  that  any  stimulus  confers  permanent  benefits  seems  to  defy 
basic  evolutionary  principles.  If  a  stimulus  is  always  present,  there 
seems  to  be  no  possible  reason  for  an  organism  to  continue  to  respond 
preferentially  to  it.  It  seems  much  more  likely  that  sequences  of  odors 
might  function  together  to  create  an  olfactory  analog  of  music  or  at  least 
of  Muzak. 

Very  little  will  happen  to  change  the  cost  structure  of  the  fragrance 
industry  in  the  near  future  because  while  better  and  cheaper 
instrumentation  is  on  its  way,  the  cost  of  the  skilled  professionals  and 
other  facilities  is  not  likely  to  go  down  in  the  big  companies.  These  firms 
are  limited  by  the  number  of  fragrances  they  can  market  and  promote  as 
much  as  by  the  number  they  can  invent.  In  today’s  world,  military 
simulation  is  perceived  to  be  a  minuscule  market.  Other  kinds  of  virtual 
reality  training,  theme  parks,  arcades,  and  home  entertainment  systems 
suggest  a  much  larger  market  but  history  is  littered  with  flash-in-the- 
pan  odor  inventions. 

The  only  bright  spot  is  the  fact  that  the  industry  is  moving  past 
fragrances  and  odorized  products  such  as  detergents  to  a  expansion  of  the 
category  of  products  intended  to  make  our  olfactory  environment  richer. 
Room  and  car  odorizing  products  are  now  standard  fare  in  grocery  stores. 
Many  of  these  products  are  representations  of  fruits  or  woods.  While  many 
are  poorly  done,  they  represent  an  appetite  on  the  part  of  the  public  that 
has  not  yet  been  sated.  However,  given  the  difficulty  of  adding  odors  to 
existing  entertainment  systems,  it  is  not  likely  that  any  major 
investment  will  be  made  in  the  private  sector  in  the  near  term.  Perhaps, 
as  large  screen  projection  television  on  a  chip  invades  the  home  in  the 
next  few  years,  theaters  and  arcades  will  feel  the  pressure  to  improve  the 
sense  of  presence  that  their  experiences  provide. 
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VI.  Simulation 


This  category  of  effort  was  expected  to  be  a  major  activity  of  the  PI  since 
it  is  the  area  in  which  he  has  the  most  experience.  However,  the 
difficulties  in  getting  the  odor  delivery,  odor  representation,  and  tracking 
problems  solved  led  to  much  more  focus  on  those  areas. 

VI.  Odor  Cue  Environment 

Three  odor  scenarios  where  modelled.  The  first  was  a  bare  virtual 
environment  furnished  with  objects  that  might  be  expected  to  smell.  The 
user’s  movements  toward  and  away  from  any  of  these  objects  led  to  an 
increasing  or  decreasing  concentration  of  the  expected  odor  stimulus.  This 
test  was  done  with  wireless  video  and  wireless  olfactory  display  but  the 
head-tracking  was  done  by  conventional  magnetic  tracking.  Thus,  the 
participant  was  tethered  to  the  computer.  This  ability  to  explore  the  odor 
environment  physically  was  much  more  effective  than  when  the  odors 
were  presented  to  a  seated  user  looking  at  the  same  visual  stimulus. 
Voluntary  physical  movement  with  respect  to  the  odor-producing  objects 
not  only  connected  the  odor  to  the  visual  stimulus  but  it  also  seemed  to 
make  the  odor  itself  seem  more  real.  It  was  also  interesting  that  when  a 
strong  odor  was  encountered,  it  was  quite  unpleasant  for  people  if  they 
could  not  reduce  it  by  moving  their  heads  away  since  that  was  their  first 
instinct.  If  it  did  not  decrease  when  they  moved  their  heads  away,  they 
felt  trapped  and  tore  off  the  mask. 

VI. B.  Traffic  Accident  Scenario 

The  second  situation  that  was  modelled  was  an  urban  accident  scene  in 
which  we  tried  to  determine  how  many  odors  would  be  required  to 
simulate  a  realistic  emergency  medical  situation.  In  this  scenario,  one  car 
operated  by  a  drunken  driver  runs  a  stop  light,  hits  another  car  and  a 
pedestrian,  comes  to  a  stop  after  knocking  over  some  garbage  cans,  and 
catches  fire.  The  driver  staggers  out  and  falls  on  the  pavement.  An 
ambulance  arrives  and  a  paramedic  performs  triage  on  the  three  victims: 
the  drunken  driver,  the  driver  of  the  other  car,  and  the  pedestrian. 

As  she  approaches  the  car  that  was  struck,  she  smells  the  exhaust  from 
the  still  running  vehicle.  She  sees  that  the  driver  is  bleeding  profusely 
from  a  superficial  head  wound.  She  smells  the  blood  and  the  sweat  from 
the  workout  he  had  before  the  accident.  She  also  smells  the  gasoline  from 
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a  ruptured  gas  tank  flowing  away  from  the  car.  Since  this  individual  is 
conscious  and  not  in  immediate  danger  and  the  fuel  appears  unlikely  to 
ignite,  she  tells  the  driver  to  turn  off  the  car  and  moves  on  to  the  next 
victim. 

As  she  approaches  the  injured  pedestrian  lying  on  the  street,  she  passes  a 
pizzeria  and  is  momentarily  immersed  in  the  aroma  that  is  wafting  from 
it.  She  can  see  that  the  injured  woman’s  abdomen  has  been  penetrated  and 
smells  that  her  bowel  has  been  ruptured.  While  this  person’s  injury  is 
severe,  she  will  be  OK  for  a  few  minutes. 

The  paramedic  then  moves  quickly  on  to  the  drunken  driver  lying  on  the 
pavement  near  his  car.  He  is  surrounded  by  a  pool  of  blood  which  is  coming 
from  arterial  bleeding  in  his  leg.  She  smells  the  blood  as  well  as  alcohol 
on  his  breath.  She  might  notice  the  smell  of  the  garbage  from  the 
overturned  trash  cans  and  will  definitely  observe  the  stench  of  the  smoke 
being  blown  her  way  intermittently  from  his  burning  vehicle.  But  since 
this  individual  has  lost  a  lot  of  blood,  she  applies  pressure  until  the 
bleeding  stops  and  quickly  dresses  the  wound  before  returning  to  the 
person  with  the  abdominal  injury  who  while  more  severely  injured  could 
afford  to  wait  the  few  minutes  required  to  save  this  man’s  life. 

This  accident  is  a  realistic  training  scenario  for  a  civilian  paramedic.  It 
involves  10  odors  and  could  therefore  be  supported  by  the  current  delivery 
system. 

VI. C.  Battlefield  Scenario 

The  final  scenario  that  was  modelled  was  a  battlefield  casualty  situation 
in  a  rural  setting.  There  are  two  vehicles  beside  a  country  road,  one  of 
which  is  burning  and  the  other  is  still  running.  The  scene  is  a  grassy 
setting  at  the  edge  of  a  pine  forest  with  a  variety  of  low  lying  evergreen 
bushes  at  the  margin. 

Again  there  are  three  casualties.  The  medic  must  decide  which  of  the 
casualties  to  treat  first.  One  has  stepped  on  a  land  mine  and  had  his  foot 
blown  off.  He  is  lying  on  his  back  with  his  foot  elevated  near  the 
overturned  earth  from  the  explosion.  He  smells  of  blood  and  sweat  from 
weeks  in  the  field.  A  second  soldier  has  a  sucking  chest  wound  from  a 
sniper’s  bullet.  A  third  is  lying  on  his  stomach.  His  back  is  badly  burned. 
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The  smell  of  burnt  flesh  is  very  pronounced  near  him. 

Again,  the  odors  required  fit  within  the  capabilities  of  the  current 
delivery  system.  Of  course,  it  would  be  possible  to  construe  scenarios  in 
which  many  more  odors  would  appear  to  be  necessary.  However,  given  the 
current  doctrine  of  stabilize  and  transport,  the  medic  will  not  do  as  much 
cleaning  of  wounds  as  he  might  have  done  50  years  ago. 

VI. D.  Simulation  Summary 

These  scenarios  are  representative  of  the  kinds  of  situations  that  might 
need  to  be  simulated.  The  importance  of  the  odors  to  simulated 
environments  has  been  observed.  It  is  possible  to  design  formal 
experiments  that  would  prove  or  disprove  the  value  of  the  odors  on  the 
training  itself.  However,  until  the  odors  needed  are  truly  convincing,  such 
critical  experiments  would  be  premature.  All  that  would  be  tested  for 
sure  would  be  the  quality  of  the  odorants  themselves  and  we  would  be 
willing  to  stipulate  that  so  far  they  are  not  very  good. 
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VII.  Odor  Studies 

There  are  three  applications  of  odors  that  are  of  interest  in  medical 
practice  and  training.  The  first  is  the  use  of  odors  to  increase  the  realism 
of  a  virtual  reality  simulation  for  training  purposes.  The  second  is  the 
more  speculative  use  of  odors  in  telesurgery.  The  third  is  the  use  of  non 
representational  odors  as  part  of  the  human  interface  during  medical 
procedures. 

VILA.  Simulation 

The  importance  of  the  odors  to  simulated  environments  has  been 
demonstrated  during  this  project  for  the  odors  that  are  available. 
However,  the  effect  of  odors  is  much  more  valuable  when  coupled  with 
head  movement  in  an  immersive  system  than  in  a  desktop  environment.  It 
may  be  that  this  is  because  the  desktop  user  is  grounded  in  the  reality  of 
the  office  environment  rather  than  the  one  depicted  on  the  screen.  A 
visually  immersive  but  sedentary  environment  might  benefit  from  odors. 
However,  when  driving  in  a  car,  not  knowing  whether  an  odor  is  coming 
from  inside  or  outside  the  vehicle  is  a  common  confusion  in  real  life, 
especially  in  those  cases  when  the  odor  might  signal  an  immediately  life 
threatening  situation. 

Vil.B.  Telesmell 

Olfactory  telepresence  is  a  very  interesting  possibility.  While  odor  is  not 
always  important  in  surgical  procedures,  there  are  situations  when  it  is 
vital.  In  these  cases,  a  remote  surgeon  would  lack  diagnostic  information 
that  would  be  obvious  if  he  was  at  the  scene.  For  instance,  the  smells  of 
feces,  urine,  or  stomach  acid  could  instantly  tell  the  remote  surgeon  what 
organs  had  been  ruptured  by  a  bullet  wound.  The  smell  of  infection  would 
instantly  tell  him  that  an  appendix  had  burst. 

During  this  project,  the  PI  spent  considerable  time  working  out  the 
technology  of  a  telesmell  demonstration  first  with  Dr.  Paul  Keller  of 
Pacific  Northwest  Labs  and  the  with  Dr.  Brian  Andreesen  at  Lawrence 
Livermore  Laboratory. 

Telesmell  depends  on  two  technologies  which  are  in  their  infancies.  The 
first  is  odor  delivery  which  was  addressed  in  this  project.  The  second  is 
artificial  nose  technology  as  has  been  developed  by  the  two  individuals 

57 


above. 


VII.B.1.  Remote  Odor  Display 

The  odor  delivery  system  developed  in  this  project  is  sufficient  for  a 
preliminary  demonstration  in  that  it  can  be  loaded  with  a  set  of  odors 
which  are  deemed  to  be  significant  for  the  purposes  of  the  demonstration 
or  even  in  a  particular  practical  application  such  as  telesurgery.  Then  it 
could  display  odors  from  that  repertoire  that  have  been  detected  in  the 
remote  location.  The  only  problems  would  be  the  inadequacy  of  the 
available  odorant  materials  pointed  to  throughout  this  report.  However,  if 
an  unexpected  odor  appeared  at  the  remote  site,  the  system  would  be 
helpless.  There  would  be  no  way  it  could  ad  lib  an  unknown  odor.  For 
instance  if  a  soldier  had  suffered  a  chemical  burn  and  that  eventuality  had 
not  been  anticipated,  the  system  would  not  be  able  to  display  it. 

VII. B. 2.  Artificial  Nose  Technology 

Detecting  the  odors  in  the  remote  site  is  the  job  of  artificial  nose 
technology  which  is  a  trendy  topic.  The  instrumentation  that  is  used  to 
identify  unknown  odors  in  the  laboratory  is  being  miniaturized.  Dr. 
Andreeson  had  a  portable  Time-of-Flight  mass  spectrometer  that  was 
originally  mounted  on  an  airplane  to  look  for  evidence  of  underground  Iraqi 
cities.  We  thought  we  could  use  this  device  to  identify  a  small  set  of 
odors  in  real  time.  In  addition,  there  are  many  research  projects  and  even 
a  few  products  that  are  using  neural  nets  to  analyze  the  responses  of  a 
number  of  sensors  used  in  concert.  These  sensors  are  often  based  on 
polymers  with  different  coatings  that  respond  differentially  to  different 
odorant  compounds.  The  sensors  change  their  resistance  or  their  resonant 
frequency  in  ways  that  are  characteristic  of  each  odor  and  their  signals 
are  fed  into  a  neural  net  (already  trained  with  known  odors)  that  then 
outputs  the  identification. 

At  the  time  the  PI  was  evaluating  this  technology,  these  sensors  took  a 
minute  and  a  half  to  respond,  far  too  slow  for  telepresence.  However,  the 
PI’S  conversations  with  Nate  Lewis  of  Cal  Tech  about  his  technology  may 
have  influenced  the  latter  to  emphasize  the  speed  of  his  sensors  which 
Cyrano  Systems,  the  company  which  was  formed  to  commercialize  his 
technology,  now  proudly  proclaims  are  capable  of  sub-second  response. 
This  performance  suggests  that  a  telesmell  system  based  on  these 
sensors  would  be  feasible. 


58 


Aromascan  Corporation  has  a  system  based  on  similar  sensors.  It  works 
like  a  laboratory  device.  The  instrument  takes  1 .5  minutes  to  identify  a 
sample  once  it  has  been  introduced,  it  is  likely  that  the  entire  process  of 
preparing  the  sample  and  removing  it  after  the  analysis  would  take 
considerably  longer.  The  Cyrano  Systems  devices  are  new  and  are  being 
marketed  as  single  purpose  sensors.  They  have  no  general  purpose  device 
that  could  continuously  sample  the  ambient  air  and  be  trained  to 
simultaneously  detect  a  wide  range  of  odorants  so  there  is  no  way  to 
predict  how  large  such  a  system  might  be. 

The  problem  with  these  chemical  sensors  in  general  is  that  they  are 
analog  devices  that  are  likely  to  be  affected  by  the  materials  that  they 
are  sensing.  Drift,  aging,  temperature  sensitivity,  contamination,  fouling, 
etc.  are  all  likely  to  be  problems.  Similarly,  the  issue  of  continuously 
sampling  the  ambient  air  is  a  new  one.  A  sensor  array  based  on  these 
devices  may  be  affected  by  what  it  was  smelling  immediately  before  as 
well  as  what  it  is  smelling  now. 

Lawrence  Livermore’s  approach  was  to  use  a  Time-of-Flight  Mass 
Spectrometer.  This  device  was  bulky  and  would  only  be  suitable  for  a 
proof  of  concept  demonstration  rather  than  practical  device.  However,  the 
miniaturization  of  mass  spectrometers  continues  apace  and  there  is 
currently  a  device  2"x2”x1”  that  suggests  that  a  portable  system  based  on 
this  approach  is  thinkable. [Design] 

V1I.B.3.  Proposed  Telesmell  Demonstration 

It  should  be  noted  that  telesmell  should  probably  start  with  a  project  that 
demonstrates  the  idea  as  part  of  a  system  before  the  ultimate 
miniaturization  and  the  detection  of  subtle  odors  is  attempted.  The 
project  the  PI  devised  with  Paul  Keller  was  to  select  a  group  of  odors  for 
which  sensors  could  developed  with  certainty  of  success.  Household 
cleaners  were  chosen  as  fitting  within  the  guaranteed  capabilities  of  the 
sensors  that  Dr.  Keller  was  already  working  with.  Then,  two  environments 
would  be  created:  the  odor  environment  and  the  remote  sensing 
environment.  The  remote  environment  would  be  an  empty  space  containing 
only  a  wide  area  wireless  tracking  system  with  wireless  audio  and  video 
image  transmission  to  a  participant  wearing  an  HMD  and  an  odor  delivery 
system. 
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As  the  participant  walked  around  the  remote  sensing  environment,  his 
position,  head  orientation,  and  direction  of  gaze  would  be  tracked 
optically.  That  information  would  be  transmitted  to  the  odor  environment 
in  which  a  second  person  would  act  as  a  robotic  cameraman  and  nose  for 
the  individual  in  the  remote  sensing  environment.  As  the  remote 
participant  moved,  the  robotic  cameraman  would  move  accordingly  and 
look  in  the  same  direction.  Audio  cues  would  tell  the  robotic  cameraman 
how  to  move  his  head  to  duplicate  the  actions  of  the  remote  participant. 
The  video  image  seen  by  the  robotic  camera  would  be  transmitted  to  the 
remote  sensing  environment  and  displayed  on  the  remote  participant’s 
HMD. 

To  stabilize  the  image  and  to  minimize  lag,  the  robotic  camera  would  have 
much  higher  resolution  than  a  standard  video  camera  as  well  as  a  wide- 
angle  lens.  Only  the  NTSC-resolution  window  needed  by  the  remote 
participant  would  be  transmitted.  While  this  remote  control  setup  would 
be  a  little  awkward,  it  would  probably  be  sufficient  for  the  task  and 
would  definitely  be  cheaper  than  a  true  robotic  camera  capable  of  slewing 
at  the  rates  of  human  movement.  In  addition,  this  master/slave 
arrangement  might  be  of  use  in  a  variety  of  teleoperation  applications 
where  an  expert  in  one  location  wants  to  direct  the  actions  of  a  non¬ 
expert  in  another. 

As  the  remote  participant  navigated  the  remote  environment,  he  would  not 
only  see  the  odor  environment  through  his  HMD,  he  would  smell  the  odors 
as  well.  The  robotic  cameraman  would  wear  an  artificial  nose  system  that 
would  inhale  the  ambient  air,  identify  one  or  more  of  a  set  of  ten 
prearranged  odors,  note  the  concentration,  and  transmit  this  information 
to  the  remote  participant  who  would  perceive  it  through  the  wireless  odor 
delivery  system  that  he  is  wearing.  (It  is  not  critical  that  the  artificial 
nose  be  comfortably  wearable  to  test  the  concept.  It  would  be  sufficient 
to  mount  it  on  an  IV  stand  on  wheels  of  the  type  that  patients  commonly 
push  around  hospitals.) 

VII.B.4.  Future  Telesmell 

If  this  preliminary  demonstration  provides  a  convincing  experience  of 
smelling  the  remote  odors,  then  the  problems  of  miniaturizing  the 
artificial  nose  and  odor  delivery  technologies  can  be  addressed.  And  even 
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more  challenging,  the  development  of  the  A-nose  components  needed  for 
particular  applications  can  be  perfected. 

Another  motivation  for  the  telesmell  capability  described  can  found  by 
collapsing  the  distance  down  to  zero.  It  is  commonly  assumed  that  at 
least  some  soldiers  will  wear  special  suits  that  protect  them  from 
biological,  chemical,  and  radiological  weapons,  in  these  cases,  they  cannot 
breathe  the  ambient  air  unless  it  is  heavily  filtered.  Odors  will  be 
removed  along  with  the  CBR  agents.  Denying  a  soldier  one  of  his  senses 
will  not  only  make  him  uncomfortable  but  will  also  cut  him  off  from  the 
immediate  battlefield.  To  increase  his  battlefield  awareness,  it  will  be 
necessary  to  have  A-nose  technology  sense  the  odors  in  his  surroundings 
and  an  odor  delivery  capability  in  the  suit  to  reconstitute  those  odors  for 
him.  When  going  to  all  this  trouble,  it  would  be  useful  to  provide  him  with 
a  magnified  odor  sense  that  will  allow  him  to  detect  and  track  the  odors 
of  enemy  soldiers,  enemy  campfires,  and  enemy  vehicles. 

Vli.C.  Odor  at  the  Human  Interface 

There  is  a  body  of  science  that  suggests  that  odors  have  an  effect  on 
human  performance  independent  of  their  power  for  realistically 
representing  real  world  environments.  These  studies  show  that  odors 
elevate  mood,  improve  memory,  and  heighten  vigilance.  If  these  results 
stand  up,  there  would  seem  to  be  many  situations  in  which  such  effects 
would  offer  practical  benefits. 

VII.C.1.  Odor  Science 

These  studies  were  originally  motivated  by  a  desire  to  subject  the  age-old 
claims  of  aroma  therapy  to  scientific  scrutiny.  And  while  the  studies  have 
been  done  by  scientists,  they  should  be  received  as  suggestive  rather  than 
authoritative.  The  reason  is  that  few  of  the  studies  involve  the  large 
numbers  of  subjects  needed  to  overwhelm  the  large  individual  differences 
in  odor  sensitivity  in  the  population.  Even  fewer  have  been  replicated  by 
other  investigators.  Standards  on  what  is  meant  by  “peppermint”  have  not 
been  established  since  this  term  can  be  applied  to  a  variety  of  materials 
produced  by  different  means. 

A  few  of  the  studies  have  experimental  flaws  which  may  taint  their 
results.  For  instance,  a  study  at  the  University  of  Cincinnati  by  Joel  Warm 
and  William  Dembers,  reports  that  odors  improved  vigilance. [Warm] 
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However,  the  odor  was  introduced  intermittently  by  a  fan  that  was  only 
turned  on  when  the  odor  was  being  displayed.  Thus,  the  sound  of  the  fan 
and  the  movement  of  the  air  always  accompanied  the  odor  cue  confusing 
the  issue  of  which  stimulus  actually  caused  the  improved  vigilance.  When 
this  issue  was  pointed  out  to  the  researchers,  they  cited  the  fact  that  it 
was  the  best  instrumentation  they  could  come  up  with  since  they  had  no 
expertise  in  electronics. 

VII.C.2.  Odor  Studies  in  This  Project 

This  project  made  two  contributions  to  this  literature.  Both  studies 
involved  90  subjects,  enough  to  make  them  unassailable  from  a  standpoint 
of  experimental  design. 

The  first  tested  the  hypothesis  that  odors  could  improve  the  spatial 
reasoning  ability  of  a  surgeon  while  he  was  operating.  This  study  was 
conducted  by  Dr.  Susan  Knasco  at  the  Monell  Chemical  Senses  Center  in 
Philadelphia. [Knasco96]  Her  study  showed  a  10%  improvement  in 
performance  on  standard  tasks  for  spatial  reasoning  at  the  .001 
significance  level  when  subjects  were  exposed  to  peppermint.  There  was  a 
smaller  effect  with  lavender  and  none  with  the  other  odors  tested.  This 
effect  in  large  enough  to  be  of  practical  as  well  as  of  scientific 
significance.  The  report  on  this  study  was  delivered  earlier  in  the 
program. 

A  second  study  tested  the  related  supposition  that  odors  might  improve 
manual  dexterity  as  well.[Knasco97]  In  this  case,  no  effect  was  noted. 
That  two  studies  on  such  closely  related  tasks  would  have  such  different 
results  suggests  either  that  the  studies  just  happened  to  find  an 
important  boundary  between  odor  effects  or  that  the  strong  result  of  the 
first  study  should  be  subjected  to  further  scrutiny.  Since  these  studies 
while  involving  a  large  number  of  subjects  did  so  only  for  a  few  hours 
apiece,  it  is  reasonable  to  be  skeptical  about  whether  there  will  be  long 
term  benefits  of  such  exposure.  Nevertheless,  even  a  one-shot 
improvement  might  provide  potential  warfighting  advantage,  if  the 
performance  of  a  MASH  unit  could  be  improved  for  just  the  24  hour  period 
surrounding  a  single  battle.  The  report  on  this  study  was  delivered  earlier 
in  the  program. 

The  very  high  cost  of  these  studies,  the  equivocal  nature  of  the  results, 
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and  the  realization  that  other  parts  of  the  program  needed  additional 
resources  led  us  to  curtail  further  studies  in  this  program.  The  most 
useful  study  in  terms  of  this  program  would  have  been  one  which  showed 
whether  odors  improved  the  efficacy  of  virtual  reality  based  training. 
However,  given  the  difficulty  in  getting  realistic  odorants  produced,  such 
a  study  would  be  premature.  It  would  be  impossible  to  separate  the  quality 
of  the  odor  from  the  effect  of  the  odor  on  the  training.  Furthermore,  such 
testing  would  require  investing  considerable  effort  in  the  development  of 
training  scenarios.  While  it  would  have  been  possible  to  test  whether 
odors  had  an  effect  on  the  participants'  sense  of  presence,  that  concept  is 
so  poorly  defined  that  the  study  would  be  reduced  to  asking  people  if  they 
thought  the  odors  enhanced  the  experience.  The  evidence  provided  by  such 
an  opinion  poll  seems  strictly  cosmetic  and  far  too  weak  to  use  as  an 
argument  for  or  against  using  odors  in  any  given  application. 
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VIII.  Recommendations 

VIII. A.  Introduction 

The  purpose  of  this  project  was  to  add  olfactory  stimuli  to  virtual  reality 
simulations.  However,  the  project  exposed  the  researchers  to  the  current 
state  of  both  olfactory  science  and  technology  as  well  as  to  the  deficits 
and  opportunities  in  these  fields.  In  the  process,  I  will  argue  that  there 
are  significant  benefits  that  would  accrue  from  defining  and  meeting  the 
following  grand  challenge:  the  invention  of  an  Olfactory  Super  Sense  which 
is  as  superior  to  our  natural  sense  of  smell  as  the  microscope  and 
telescope  are  to  our  natural  vision.  Finally,  we  will  comment  on  some 
other  weaknesses  of  current  technology  in  other  areas  that  were 
confronted  during  this  project. 

VIII. B.  Olfactory  Science 

The  chemical  senses  of  smell  and  taste  are  the  oldest  in  evolutionary 
terms  and  represent  a  significant  fraction  of  both  our  genetic  code  and  our 
brain  mass.  And  yet,  the  scientific  understanding  of  olfaction  is  crude 
compared  to  our  knowledge  of  vision  and  hearing.  Given  the  extent  to 
which  animal  behavior  can  be  ruled  by  smell  and  the  evidence  that  we  are 
also  subject  to  unconscious  odor  influences,  it  is  simply  good  science  to 
redress  this  imbalance  by  raising  our  level  of  understanding  of  these 
senses  to  that  of  vision  and  hearing. 

We  should  know  how  we  smell,  what  we  smell,  how  to  duplicate  smells, 
and  how  to  deliver  them.  While  there  is  a  considerable  amount  of  study 
going  on,  it  is  fragmented,  preliminary,  narrowly  focused,  and  seldom  asks 
the  questions  that  immediately  arise  when  considering  practical 
applications.  Odor  scientists  have  dwelled  upon  behavioral  responses  to  a 
small  set  of  simple  odors  in  the  laboratory.  They  test  odor  thresholds 
against  a  background  of  no  odor  even  though  in  real  life  we  are  typically 
confronted  with  ever  changing  concentrations  of  smells  against  an 
indeterminant  background. 

VIII. B.1.  Human  Olfactory  Capabilities 

Children  and  adults  are  routinely  given  eye  exams  which  give  us  very  good 
information  about  the  performance  of  the  population  on  the  vision 
abilities  needed  for  reading  and  driving.  Hearing  tests  are  also  common, 
especially  when  any  deficit  in  understanding  language  is  observed. 
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However,  olfaction  is  not  thought  to  be  critical  to  everyday  life  and  is 
only  measured  when  a  complete  deficit  is  noticed  after  an  illness  or 
injury  or  when  a  person  complains  of  a  persistent  malodor  not  perceptible 
to  those  around  him.  As  a  consequence,  our  knowledge  of  how  well  people 
can  smell  is  much  more  poorly  developed.  We  know  that  women  are  more 
sensitive  than  men  and  young  people  are  more  capable  than  the  elderly.  We 
also  know  that  there  are  enormous  differences  (up  to  two  orders  of 
magnitude)  among  individuals  of  both  sexes. 

The  problem  is  aggravated  by  the  fact  that  we  do  not  understand  the 
fundamental  mechanisms  of  smell  and  the  possibility  that  olfaction  is  not 
based  on  a  neat  organizing  principle  such  as  the  spectra  of  light  or  sound. 
In  vision,  we  can  easily  understand  myopia  in  terms  of  optics,  color 
blindness  in  terms  of  the  cones,  and  poor  contrast  perception  in  terms  of 
aging  rods.  In  hearing,  we  can  understand  selective  frequency  deficits  in 
terms  of  broken  cilia  in  the  inner  ear. 

We  know  that  people  can  perform  remarkably  sophisticated  feats  of  odor 
discrimination.  For  instance,  woman  living  together  use  their  sense  of 
smell  to  synchronize  their  menstrual  periods.  Women  can  identify  their 
husbands’  laundry  by  smell  alone.  Women  can  measure  genetic  distance  and 
find  the  scent  of  unrelated  men  more  pleasant  than  that  of  kin,  except 
when  they  are  pregnant  and  the  preference  is  reversed. [Richardson] 
Similarly,  both  sexes  can  determine  the  physical  attractiveness  of  a 
member  of  the  opposite  sex  by  smell  alone. [Mundell]  Men  find  the  scent  of 
attractive  women  pleasant  whereas  women  find  the  scent  of  attractive 
men  unpleasant. 

The  fact  that  we  are  not  often  aware  of  such  perceptions  makes  it  even 
more  important  to  understand  them  because  we  do  not  know  how  often 
such  subconscious  factors  may  be  influencing  our  behavior.  It  may 
determine  who  we  get  along  with  and  why  certain  people  instinctively 
dislike  each  other.  This  could  be  significant  in  team  building.  Given  that 
different  ethnic  groups  certainly  have  different  odor  characteristics 
based  on  their  diets  and  possibly  because  of  different  genetic  makeups, 
unconscious  odor  judgments  should  be  considered  a  possible  factor  in  any 
situation  where  different  nationalities  and  ethnic  groups  must  cooperate 
such  as  in  peace  keeping  missions  or  diplomacy. 


65 


These  observations  suggest  that  our  sense  of  smell  is  more  powerful  than 
we  often  assume  and  that  we  may  have  capabilities  that  we  do  not  use.  We 
need  to  understand  how  well  the  population  can  smell  things.  We  need  to 
know  what  odors  each  individual  is  blind  to  and  why.  We  need  to  settle  the 
issue  of  nature  versus  nurture.  To  what  extent  is  the  ability  to  smell  fixed 
at  birth  and  to  what  extent  is  it  learned  through  experience?  It  is  further 
important  to  understand  how  much  that  learning  can  be  improved  with 
training.  We  know  that  some  people  are  super  smellers.  But  is  that 
because  they  have  more  receptors,  more  sensitive  receptors,  additional 
types  of  receptors,  or  do  they  have  more  processing  in  the  olfactory  bulb? 
We  believe  that  people  are  very  poor  at  recognizing  the  ingredients  in 
mixtures.  Is  that  just  because  we  are  seldom  asked  to  do  so  or  is  it  a 
fundamental  consequence  of  how  olfaction  works? 

We  see  it  written  that  humans  can  distinguish  10,000  different  odors,  but 
it  is  not  clear  what  that  means.  Many  people  can  identify  only  a  few  odors 
reliably.  They  often  will  be  stumped  by  even  the  most  familiar  smells. 
Furthermore,  they  can  be  convinced  that  one  smell  is  another  quite  easily. 
While  professional  perfumers  indeed  have  a  working  knowledge  of  many 
odorous  chemicals,  they  do  not  report  that  their  everyday  olfactory 
experience  is  that  different  than  anyone  else’s.  It  would  be  useful  to  see 
how  many  odors  ordinary  people  can  identify  in  isolation  and  how  many 
they  can  correctly  identify  from  a  small  set  of  candidate  odors. 

This  information  may  reveal  diseases  and  genetic  conditions  that 
heretofore  escaped  notice.  It  may  also  explain  occupational  preferences 
and  performance.  For  instance,  a  surgeon  may  miss  a  clue  because  he  is 
unable  to  smell  it.  We  know  for  instance  that  older  people  lose  their 
ability  to  smell  mercaptans  that  are  added  to  give  natural  gas  an  odor. 
Clearly,  a  new  alarm  odor  is  needed  given  the  aging  population. 

Given  that  we  live  in  a  society  in  which  identifying  odors  is  almost  never 
a  matter  of  survival  and  seldom  of  any  practical  significance  whatsoever, 
it  would  be  valuable  to  study  the  sensorium  of  primitive  people  such  as 
the  natives  of  Papua  New  Guinea  who  often  smell  things  before  they  see 
them  in  the  rain  forest  and  whose  culture  is  based  much  more  on  smell 
than  ours.  It  is  possible  that  they  have  more  intact  odor  sensing  systems 
because  they  use  them,  just  as  the  wild  creatures  from  which  house  cats 
are  thought  to  have  originated  have  larger  brains  than  their  domesticated 
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descendants. 


VIII. B.2.  Genetic  Information 

In  the  past,  we  knew  little  about  the  genetic  basis  for  odor  perception 
other  than  it  constituted  a  surprisingly  large  proportion  of  our  genetic 
endowment.  During  the  period  of  this  project,  that  knowledge  has  been 
greatly  expanded  by  the  identification  of  1000  genes  that  appear  to  code 
for  the  different  kinds  of  receptors.[Glausiusz]  Of  these,  about  30%  appear 
to  be  defective— probably  because  olfaction  no  longer  figures  strongly  in 
human  survival.  In  fact,  they  may  have  started  to  atrophy  the  moment  we 
walked  upright  because  we  get  much  less  information  from  our  nose  in  the 
air  than  we  did  when  we  kept  it  close  to  the  ground. 

It  is  now  possible  to  think  about  identifying  the  function  of  these 
receptors.  Initially,  we  may  well  find  that  there  is  a  genetic  basis  for 
some  specific  anosmias  (odor  blindnesses).  When  there  is  not,  we  can  look 
for  environmental  or  experiential  factors  that  may  explain  such  a 
selective  loss  of  function. 

We  will  want  to  identify  the  odor  function  of  each  gene  and  therefore  each 
type  of  receptor.  What  odors  can  the  receptor  detect.  How  does  it 
function?  Does  it  react  to  a  particular  chemical,  to  the  size  and  shape  of  a 
molecule,  to  the  strength  of  different  bonds,  etc?  How  does  each  receptor 
react  to  each  chemical?  Does  a  single  chemical  as  opposed  to  a  complex 
odor  affect  more  than  one  receptor?  How  does  each  receptor  react  to  each 
complex  odor?  This  information  will  help  us  determine  which  ingredients 
of  complex  odors  are  most  significant  in  the  odor  sensation  they  evoke.  It 
may  also  alert  us  to  olfactory  homonyms,  chemically  different  substances 
that  smell  alike.  In  fact  even  if  there  are  not  primary  odors,  there  may  be 
an  odor  alphabet  from  which  many  complex  odors  can  be  formed. 

We  will  want  to  screen  the  population  to  see  how  the  different  receptors 
are  distributed  among  us.  How  much  do  we  differ  in  our  ability  to  smell 
different  things?  How  much  of  that  distribution  is  based  on  heritage  and 
how  much  on  random  variation?  In  particular,  we  will  want  to  find  out 
what  genetic  traits  distinguish  the  super  smellers  from  the  rest  of  us.  We 
may  want  to  screen  for  super  smellers  and  to  see  if  there  are  tasks  for 
which  their  capabilities  uniquely  suit  them  such  as  tracking. 
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Understanding  these  mechanisms  may  lead  us  to  strategies  that  we  can 
use  to  improve  our  instrumentation  which  still  lags  natural  technology.  It 
will  also  help  us  in  identifying  strategies  that  can  be  employed  in  future 
delivery  system  design  and  odorant  synthesis. 

Vill.B.3.  Animal  Capabilities 

At  the  same  time,  we  have  to  understand  animal  olfactory  capabilities 
much  better  than  we  do.  We  need  to  know  whether  animals  are  more 
sensitive  because  they  have  their  noses  to  the  ground,  because  odors  are 
needed  for  their  survival,  because  they  have  more  receptors,  because  the 
have  more  surface  area  in  their  olfactory  epithelia,  or  because  they  have 
types  of  receptors  that  humans  do  not.  Something  is  known  about  which 
creatures  have  the  best  sense  of  smell,  but  it  would  be  useful  to  anyone 
studying  them  to  know  what  they  can  smell,  how  sensitive  they  are  to  it, 
and  how  each  odor  is  presented  in  their  environments.  This  information 
could  lead  to  the  identification  of  unique  capabilities  in  some  animals  and 
perhaps  motivate  us  to  try  to  duplicate  them. 

An  evaluation  should  also  be  performed  on  different  techniques  that  some 
creatures  employ  for  acquiring  chemical  information  about  their 
environments.  For  instance,  kiwi  birds  have  their  nostrils  near  the  tips  of 
their  beaks  so  they  smell  what  is  under  the  leaves  in  the  forest  or  the 
sand  on  the  beach.  Snakes  use  forked  tongues  to  sample  the  air  and  the 
detritus  on  the  ground.  They  then  place  the  tips  in  a  special  receptacle  in 
rear  of  their  mouths  where  the  actual  chemical  sensors  are  located.  The 
two  prongs  provide  a  kind  of  stereo  perception  which  helps  snakes  follow 
odor  trails.[Schwenk]  Similarly,  moths  have  elaborate  antennae  which  are 
likely  involved  in  tracking  pheromone  plumes  to  their  sources. [Taylor]  In 
these  and  similar  cases,  natural  strategies  may  suggest  techniques  that 
can  be  employed  by  human  technology.  In  general,  every  animal  sensory 
modality  should  be  identified  and  evaluated  for  duplication  for  practical 
as  well  as  scientific  reasons. 

VIII. B.4.  Odor  Environment 

In  order  to  understand  the  sense  of  smell  in  humans  and  animals,  we  need 
to  analyze  the  odor  environment  they  inhabit.  Surprisingly,  little  is  known 
about  our  everyday  olfactory  world.  We  do  not  know  what  odors  we 
encounter  and  what  odors  we  notice  as  we  go  about  our  lives.  As 
mentioned  earlier,  smelling  is  like  dreaming;  everybody  does  it  but  most 
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people  do  not  remember  doing  it.  In  extreme  situations  when  there  are 
complaints,  environmental  firms  will  be  brought  in  to  measure  the  extent 
of  noxious  odors  emanating  from  a  hog  farm  or  some  other  olfactory 
nuisance.  However,  we  know  little  about  how  odors  are  distributed  through 
the  work-a-day  world.  Do  odors  change  throughout  the  day?  Do  odors  vary 
with  altitude,  weather,  humidity,  season,  etc.  How  is  our  ability  to  smell 
affected  by  the  wind?  On  the  one  hand,  we  would  quickly  habituate  to  any 
odors  present  in  stagnant  air  and  breezes  bring  us  new  smells.  On  the 
other  hand,  high  winds  may  make  it  difficult  to  smell  at  all. 

What  is  known  about  the  sense  of  smell  has  primarily  been  learned  in  the 
laboratory.  Odors  are  presented  and  evacuated.  In  some  cases,  the 
investigators  rule  out  any  background  odor  by  prefiltering  the  air  before  it 
is  odorized.  However,  in  most  real  environments  there  is  a  background. 
How  does  our  perception  of  odors  vary  with  the  background?  We  know  that 
one  odor  can  mask  another;  however,  if  the  masking  odor  was  part  of  the 
background  we  would  already  be  habituated  to  it  and  only  smell  the  odor 
that  would  ordinarily  be  masked  resulting  in  a  totally  different 
experience. 

We  need  to  be  able  to  measure  the  odor  background  much  better  than  we 
can  now.  We  need  to  determine  whether  people  are  aware  of  the  presence 
or  absence  of  background  odors.  Special  forces  people  that  the  PI  has 
spoken  with  report  following  the  absence  of  a  smell  when  they  are 
tracking  a  person.  This  actually  makes  sense  if  in  becoming  habituated  to 
person’s  scent,  the  tracker’s  perception  of  other  odors  that  he  encounters 
is  altered  from  what  he  expects.  This  deviation  would  tell  him  that  he  is 
still  on  the  trail.  When  he  leaves  it,  his  perception  will  return  to  normal 
and  he  will  notice  that  previously  masked  elements  are  now  observable. 
Such  sophisticated  olfactory  perception  is  totally  foreign  to  most  of  us. 
However,  if  such  a  capability  exists  we  should  study  and  understand  it. 

We  know  that  sensory  deprivation  can  be  disturbing.  The  PI  has  long 
believed  that  selective  sensory  deprivation  is  also  possible.  We  have 
sensory  deprivation  environments  and  we  have  anechoic  chambers  which 
eliminate  background  sounds.  Clean  rooms  used  for  semiconductor 
manufacture  are  probably  odorless  although  their  designers  are  more 
concerned  with  particles  than  with  vapors. 
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However,  we  have  not  systematically  studied  the  effects  of  olfactory 
deprivation  except  in  those  who  have  completely  lost  their  sense  of  smell. 
One  anecdote  that  suggests  that  odor  deprivation  can  be  a  source  of 
discomfort  was  given  to  the  PI  by  a  reporter  for  a  biology  magazine  who 
had  spent  several  months  in  Antarctica.  (Because  of  the  cold,  the  air 
carries  almost  no  odors.  The  local  fauna  do  not  smell  as  creatures 
elsewhere  do  because  there  is  no  point.)  When  she  returned,  she  was 
starved  for  olfactory  stimulation  and  just  lay  on  the  ground  smelling  the 
vegetation.  It  is  likely  that  there  are  many  situations  in  modern  life 
where  we  have  so  successfully  eliminated  smells  that  one  could  argue 
that  the  addition  of  odors  would  increase  comfort  and  performance 
without  claiming  that  odors  have  magical  powers. 

In  addition,  we  can  only  understand  animal  behavior  if  we  understand  the 
odor  environments  that  they  inhabit.  It  is  not  enough  to  simply  say  they 
have  a  better  sense  of  smell  than  we  do.  What  smells  do  they  encounter, 
where  do  they  run  into  them,  and  how  do  they  affect  their  behavior? 
Because  most  mammals  are  quadrupeds,  their  noses  are  close  to  the 
ground.  If  they  are  tracking  another  animal,  its  tracks,  its  droppings,  and 
its  urine  are  all  on  or  close  to  the  ground.  Some  of  the  food  of  animals 
such  as  pigs  and  skunks  is  under  the  ground.  Heavier  molecules  from  odors 
that  are  released  in  the  air  settle  to  the  ground  and  those  from  material 
on  the  ground  never  rise  much  above  it.  How  do  these  odors  behave  over 
time?  In  general,  what  do  animals  have  available  to  smell?  What  are  the 
signals  and  what  is  the  noise? 

VIII.C.  Olfactory  Super  Sense 

In  addition  to  accomplishing  the  scientific  goals  just  laid  out,  we  also 
need  to  set  and  reach  a  corresponding  technological  goal.  Progress  in 
science,  technology,  and  industry  has  repeatedly  been  sparked  by  advances 
in  our  ability  to  perceive  natural  and  artificial  phenomena.  To  accomplish 
that  goal,  we  have  created  a  series  of  technological  senses  that  far 
exceed  our  human  endowment.  We  can  see  the  very  small  and  the  very 
distant.  We  can  see  in  the  infrared  and  the  ultraviolet.  Our  instruments 
can  detect  infrasound  and  ultrasound  in  amplitudes  far  below  our  human 
hearing  and  at  distances  far  beyond  it.  Thus,  all  that  is  being  proposed  is 
to  create  an  olfactory  supersense  that  is  akin  to  our  other  technological 
senses.  At  the  very  least,  we  should  be  able  to  smell  as  well  as  any  human 
or  any  animal.  If  we  could  do  as  well  at  detection  as  natural  creatures,  we 
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might  do  far  better  at  identification  because  unlike  them  we  would  not 
have  to  learn  from  experience.  The  entire  repertoire  of  identified 
compounds  would  be  instantly  available. 

When  there  is  a  void  in  science  and  technology,  it  is  possible  to  argue  as  a 
matter  of  faith  that  filling  the  void  will  lead  to  practical  benefits. 
However,  in  this  case  it  is  also  possible  to  identify  a  number  of  very 
specific  problems  which  might  be  addressed  by  the  fruits  of  such  a 
scientific  and  technological  endeavor.  For  the  moment,  we  will  confine 
ourselves  to  the  areas  of  interest  from  which  this  project  itself  would 
have  benefited. 

VIII. B.1.  Medical  Benefits 

Odors  have  a  long  association  with  medicine.  At  one  time,  odors  were 
thought  to  be  the  cause  of  disease  and  noblemen  had  a  servant  carrying  a 
device  to  perfume  the  air  walk  in  front  of  them.  As  recently  as  a  hundred 
years  ago,  the  nose  was  the  only  diagnostic  instrument  available  to  the 
physician.  A  number  of  common  ailments  were  known  to  have 
characteristic  odors.  Measles,  ectopic  pregnancy,  schizophrenia,  and 
sometimes  cancer  could  be  diagnosed  just  by  smelling.  Diabetes  was 
detected  by  tasting  the  urine.  In  addition,  the  body’s  reaction  to  poisons 
produces  odors  which  indicate  what  has  been  ingested. 

Today,  we  would  assume  that  these  diagnoses  are  done  in  laboratory  tests. 
However  even  there,  technicians  are  taught  to  recognize  the  different 
bacteria  that  are  being  cultured  by  smell. 

We  also  know  that  animals  and  instruments  can  make  diagnoses  of 
diseases  from  odor  alone.  When  the  PI  gave  his  first  talk  on  odors  in 
virtual  reality  at  Medicine  Meets  Virtual  Reality  in  1995,  he  asked  that 
any  doctors  in  the  audience  who  had  specific  experiences  in  which  odors 
had  been  diagnostic  in  their  practice  report  those  anecdotes  to  him.  One 
individual  (a  urologist)  described  an  incident  in  which  a  woman 
complained  to  him  that  her  dog  kept  sniffing  her  left  breast.  Upon 
examination,  he  discovered  that  she  indeed  had  a  lump  which  was  later 
determined  to  be  malignant.  Subsequent  to  that,  a  doctor  in  Florida  trained 
a  dog  to  detect  skin  cancer.[Higgins]  In  another  study,  rats  were  able  to 
detect  the  presence  of  a  cancer-producing  retrovirus  which  had  not  yet 
been  expressed  as  a  cancer  in  another  individual  by  smell 
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alone.[Beauchamp] 

Similar  results  have  been  achieved  with  instrumentation  analyzing  a 
patient’s  breath.  Dr.  Barry  Marshall  of  Australia  has  developed  a  test  to 
detect  the  ulcer  causing  Helicobacter  Pylori  from  a  patient's 
breath. [Marshall]  The  bacteria  produce  an  enzyme  called  urease.  Carbon  14 
reacts  with  urease  in  the  stomach  and  can  be  detected  in  C02  that  is 
exhaled.  Another  doctor  was  able  to  detect  Pneumocystic  Garni,  a 
complication  of  AIDS,  from  exhaled  breath. 

These  results  suggest  that  odors  contain  a  wealth  about  infections  and  the 
functioning  of  a  person’s  body,  it  is  possible  that  a  wide  range  of 
conditions  could  be  identified  by  smell.  If  so,  and  if  a  technological  device 
could  be  miniaturized  and  made  disposable,  such  tests  could  be  done  in  the 
doctor’s  office  in  a  single  visit.  This  would  eliminate  the  second  visit  to 
get  the  results,  the  handling,  the  elapsed  time,  and  the  anguish  of  people 
who  are  waiting  for  results.  Many  women  spend  a  significant  fraction  of 
their  lives  anxiously  waiting  for  false  positives  from  one  test  to  be 
overruled  by  more  sophisticated  follow-up  tests  and  biopsies. 

It  is  also  possible  that  the  health  of  the  body’s  organs  and  systems  is 
revealed  in  the  person’s  body  odor.  Given  that  body  odor  is  affected  by  diet 
and  especially  by  fat  intake,  cholesterol  and  triglyceride  levels  may  be 
evident  as  well. 

In  addition  to  medically  significant  odors,  it  is  likely  that  emotional 
states  also  modulate  a  person’s  body  odor.  Such  cues  might  be  valuable  in 
psychiatric  diagnosis  and  therapy.  They  might  help  measure  the  emotional 
state  of  patients  whose  pulse  and  blood  pressure  react  to  the  stress  of 
the  doctor’s  office. 

Even  before  embarking  on  a  technological  development,  it  would  be 
worthwhile  to  use  animals  to  determine  what  conditions  inside  the  body 
are  signaled  by  smells  that  are  easily  available  outside  of  it.  While  rats 
and  dogs  are  not  likely  to  be  welcomed  on  hospital  staffs  any  time  soon, 
their  capabilities  would  demonstrate  what  is  possible  and  motivate  the 
development  of  artificial  technologies  which  duplicate  or  surpass  them. 

Viil.C.2.  Military  Super  Sense  Applications 
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There  are  also  a  number  of  uses  that  the  military  and  law  enforcement 
officials  could  make  of  an  artificial  nose  that  exceeded  our  own.  The 
obvious  ones  would  be  in  the  detection  of  chemical  and  biological  warfare 
agents.  It  would  also  be  possible  to  detect  the  presence  of  remote 
vehicles  and  campfires  as  well  as  the  paths  used  by  enemy  patrols. 

Of  even  greater  currency  is  the  detection  of  mines,  booby  traps,  and  enemy 
personnel  in  current  and  past  battlefields  as  well  as  the  interdiction  of 
terrorists’  explosives  and  illegal  substances.  Many  approaches  to  these 
problems  are  being  considered.  One  of  the  most  effective  of  these  is  the 
use  of  dogs  which  are  known  for  their  keen  sense  of  smell  and  which  were 
used  to  good  effect  in  Viet  Nam  where  they  completely  shut  down  the 
infiltration  of  the  Viet  Cong  in  the  Delta  region.  (The  dogs  could  detect  the 
breath  exhaled  by  VC  who  were  submerged  in  the  water  and  breathing 
through  tubes.  They  could  also  hear  the  ultrasonic  sounds  generated  by  the 
wind  vibrating  trip  wires  for  booby  traps.) 

The  problem  with  dogs  in  mine  detection  is  that  they  fatigue  easily.  They 
can  only  work  for  15  minutes  at  a  time  before  they  need  a  rest.  In  general, 
they  are  trained  to  recognize  a  single  odor.  They  can  track  one  person  but 
not  look  for  any  one  of  ten.  They  cannot  look  for  mines  one  day  and 
narcotics  the  next.  Even  when  just  searching  for  mines,  dogs  have  to  be 
trained  to  each  particular  type  of  soil  and  require  two  days  to  adapt  to 
it. [Mine] 

There  would  be  a  tremendous  appetite  for  a  technology  that  could 
duplicate  canine  olfactory  performance  which  would  not  have  these 
performance  issues.  For  instance,  a  dog  can  track  a  person  given  a  sample 
of  that  person’s  odor,  but  in  the  case  where  a  sniper  is  known  to  have  shot 
from  a  particular  location,  the  dog  may  not  understand  that  you  want  him 
to  follow  the  human  scent  that  he  perceives  there.  A  synthetic  instrument 
might  be  able  to  detect  the  presence  of  a  human  scent  and  possibly  even 
recognize  the  most  recent  of  several  scents.  Unlike  the  dog,  an  instrument 
could  record  what  it  had  detected  and  transmit  that  information  anywhere 
in  the  world  to  be  tested  against  suspects  found  years  later. 

Such  a  capability  would  be  extremely  valuable  in  guerrilla  conflicts, 
peacekeeping  missions,  and  civilian  law  enforcement.  Given  that  identity 
and  genetic  distance  can  be  inferred  from  odors,  it  may  be  possible  to 
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correlate  odors  with  blood  samples,  to  recognize  blood  types  and 
hereditary  conditions.  Odor  prints  could  become  a  powerful  forensic  tool. 
If  emotional  states  are  revealed  in  a  person’s  body  odor,  they  may  be 
useful  when  interrogating  prisoners  of  war  and  criminal  suspects. 
Emotional  states  revealed  by  odors  might  also  provide  an  edge  in 
negotiations  and  diplomacy. 

Once  miniaturized,  such  devices  could  be  worn  on  the  feet  where  the 
signal  is  strongest  and  be  used  for  tracking  enemy  personnel  and  for 
warning  of  land  mines.  They  could  be  mounted  on  airplanes  and 
surveillance  drones.  A  number  of  devices  worn  by  different  individuals 
could  be  used  in  concert  to  improve  tracking  performance  just  as  a  snake 
uses  the  stereo  information  provided  by  its  forked  tongue  to  follow  the 
trail  of  its  prey. 

A  more  speculative  application  is  based  on  the  fact  that  homing  pigeons 
apparently  memorize  the  smells  of  the  terrain  they  pass  through  when 
they  are  transported  blindfolded  to  a  remote  location  and  play  back  the 
odor  sequence  in  reverse  to  find  their  way  home.  This  implies  that  there 
are  sufficient  odor  cues  to  guide  a  missile  to  a  remote  site.[Able] 

VIII. C.2.  Civilian  Super  Sense  Applications 

There  are  also  many  industrial  processes  and  consumer  applications  that 
would  be  found  for  an  improved  ability  to  detect  and  analyze  odors.  Food 
and  beverage  quality  and  freshness  control  are  obvious  possibilities  that 
are  already  being  addressed  with  today’s  mediocre  technology.  The  reason 
is  that  the  human  noses  that  have  controlled  these  processes  in  the  past 
are  getting  old  and  younger  people  have  not  shown  any  interest  in  picking 
up  the  skills.  In  addition,  the  right  instrument  would  be  more  repeatable 
and  not  be  affected  by  colds  and  vacations. 

VIII. D.  Odor  Synthesis 

The  premise  of  this  project  was  that  odors  would  be  a  necessary 
ingredient  in  simulating  real  situations.  However,  the  state  of  current 
odor  synthesis  technology  and  commerce  is  such  that  realistic  odor 
representation  should  not  even  be  attempted  without  a  budget  for  odor 
synthesis  twice  as  great  as  that  for  this  entire  project.  Even  then,  many 
of  the  odors  that  the  PI  has  been  shown  as  serious  efforts  by  those  in  the 
big  leagues  of  the  fragrance  industry  have  been  completely  unconvincing. 


Such  an  effort  would  require  a  professional  perfumer  full  time,  an 
analytical  laboratory,  and  an  inventory  of  odorous  chemicals.  Even  with 
these  resources  dedicated  to  such  a  project,  it  is  clear  that  today’s 
instruments  are  not  as  good  as  the  human  nose  that  must  be  satisfied  with 
the  results  and  that  such  an  effort  may  be  doomed  to  failure.  Therefore, 
odor  simulation  is  another  motivation  for  the  Olfactory  Super  Sense 
described  above  as  well  as  for  the  scientific  effort  required  to  have  a 
much  better  understanding  of  our  olfactory  sense  than  we  have  today. 

However,  rather  than  being  motivated  by  commercial  considerations, 
experimental  odor  synthesis  should  not  confine  itself  to  the  most 
predominant  peaks  in  the  analysis  but  should  attempt  an  exhaustive 
duplication  of  the  original  material.  To  be  repeatable  over  a  long  period  of 
time,  it  should  start  from  pure  chemical  feedstocks. 

It  may  be  that  such  an  approach  is  doomed  to  failure  just  as  sound 
synthesis  abandoned  purely  digital  synthesis  in  favor  of  sampling 
techniques  that  started  with  rich  real  world  sounds.  Or  it  might  be  that 
some  of  the  ingredients  are  not  commercially  available  or  have  not  been 
analyzed  or  synthesized  at  all.  It  would  be  useful  to  know. 

One  alternative  would  be  an  artistic  approach  which  developed  a 
repertoire  of  stock  odors  that  could  substitute  for  some  of  the  exotic 
materials.  People  are  not  very  good  at  naming  odors  and  can  be  fooled  into 
believing  that  one  odor  is  another.  However,  they  may  have  an  acute  sense 
of  what  does  not  smell  like  any  real  odor  at  all.  It  is  that  reaction  that  the 
PI  has  had  to  some  of  the  professionally  produced  materials  he  has  been 
shown. 

Another  approach  would  be  an  extension  of  the  oldest  one.  Elsewhere  in 
this  report,  it  was  mentioned  that  the  most  convincing  simulation  would 
be  accomplished  by  using  real  materials.  While  this  is  not  very  appealing 
in  the  case  of  medical  odors  and  not  very  practical  because  the  raw 
materials  lose  their  odor  quickly,  the  fundamental  processes  of 
traditional  perfumery  capture  the  odorous  compounds  from  natural 
materials.  For  instance  in  effleurage,  flower  petals  are  mixed  in  heated 
liquid  lard  which  takes  up  their  odor  producing  oils.  Then,  the  lard  is 
heated  to  the  point  where  these  oils  vaporize.  After  they  boil  off,  the 
vapor  is  fed  through  a  distillation  coil  which  is  cooled  to  condense  the 
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oils  at  its  output.  It  is  possible  that  this  process  can  be  used  to  produce 
some  of  the  odors  needed  for  realistic  simulations.  We  experimented  with 
this  process  without  great  success,  but  it  is  likely  that  real  chemists 
would  have  more  luck.  However,  it  may  be  that  many  of  the  odors  do  not 
lend  themselves  to  this  kind  of  processing. 

VIII.D.1.  Odor  Language 

One  benefit  of  a  better  understanding  of  the  olfactory  sense  and  how  it 
interacts  with  specific  chemicals  is  a  more  rigorous  odor  vocabulary. 
Currently,  there  are  a  number  of  adjectives  that  odor  professionals  use  in 
describing  the  qualities  of  a  particular  odor:  fruity,  woody,  floral, 
pungent,  musty,  etc.  However,  when  the  PI  showed  some  of  the  chemicals 
that  had  been  developed  for  him  to  other  people  in  the  industry,  they 
trotted  out  subsets  of  these  descriptors  seemingly  at  random.  There  are 
certainly  no  rigorous  standards  by  which  these  terms  are  applied  and  as  a 
result  much  conversation  about  the  subject  is  almost  meaningless. 

While  those  in  the  industry  would  disagree  with  this  characterization  and 
attribute  it  to  the  Pi’s  ignorance  and  frustration,  there  is  no  doubt  that 
this  vocabulary  is  not  grounded  in  scientific  objectivity  and  until  it  is, 
scientific  and  technological  progress  are  going  to  be  stymied.  It  would  be 
a  real  contribution  to  the  science,  technology,  and  business  of  olfaction  if 
these  words  could  be  given  an  unambiguous  scientific  footing  or  if  new 
vocabulary  based  on  scientific  knowledge  of  the  olfactory  sense  can  be 
created.  Indeed,  it  would  be  reasonable  for  the  National  Bureau  of 
Standards  to  undertake  a  preliminary  study  of  this  issue. 

If  such  concrete  vocabulary  can  be  arrived  at,  it  would  also  benefit  the 
odor  simulation  process  because  combinations  of  attributes  may  be 
duplicated  rather  than  exact  odors.  In  particular,  the  trigeminal  sense 
needs  to  be  better  understood  to  see  to  what  extent  trigeminal  sensation 
differs  from  one  odorous  compound  to  the  next.  This  could  be  done  by 
involving  those  who  have  lost  their  sense  of  smell.  Since  the  trigeminal 
sense  is  connected  to  the  brain  through  a  completely  different  cranial 
nerve,  it  is  likely  to  still  be  intact  and  functioning  in  isolation  from 
olfaction.  We  can  then  test  these  individuals  to  see  what  odors  they  can 
distinguish  from  their  trigeminal  components  alone.  This  will  tell  us  how 
many  kinds  of  trigeminal  sensation  there  are.  Ideally,  we  could  find  safe 
odorless  compounds  that  would  trigger  every  discriminable  trigeminal 
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sensation.  This  would  allow  us  to  substitute  one  trigeminal  stimulus  for 
another  when  formulating  odorants  with  the  confidence  that  we  know 
what  we  are  doing.  Such  substitution  would  be  particularly  welcome  when 
the  material  being  simulated  is  toxic. 

The  concern  about  toxicity  also  has  to  be  put  on  a  rational  footing. 
Military  training  and  military  operations  accept  a  higher  level  of  risk  than 
civilian  life.  Current  cautions  about  hazardous  substances  seem  to  have 
been  created  by  insurance  companies  wanting  to  avoid  all  risk  on  the  one 
hand  and  trial  lawyers  wanting  to  call  any  risk  actionable  on  the  other. 
Certainly,  guidelines  that  declare  the  exposure  experienced  from  standing 
on  a  street  corner  in  any  city  an  unacceptable  risk  are  irrational, 
especially  when  the  exposure  times  are  very  brief.  Declaring  everything 
toxic  makes  it  difficult  to  distinguish  the  merely  unpleasant  from  the 
truly  lethal. 

VIII.E.  Odor  Delivery 

This  project  focused  on  the  development  of  odor  delivery  systems  for 
virtual  reality  simulations  and  the  system  developed  is  a  reasonable 
contribution  towards  that  goal,  especially  because  it  is  designed  to  be 
compatible  with  the  three  major  virtual  reality  formats:  desktop, 
ambulatory,  and  CAVE.  However,  it  does  not  deal  with  all  aspects  of  odor 
perception.  Real  odor  sensation  is  often  associated  with  particles, 
droplets,  humidity,  moisture,  and  temperature  when  these  occur  in  the 
real  world.  Simulating  these  aspects  of  odor  perception  has  to  be  tested 
to  determine  their  importance  and  they  must  be  included  in  the  delivery 
system  if  they  prove  to  be  valuable. 

Similarly,  new  science  will  continue  to  shed  light  on  odor  perception  and 
possibly  add  new  requirements  to  the  delivery  system.  For  instance 
throughout  the  program,  the  PI  was  told  that  odors  could  hot  be 
lateralized  between  the  two  nostrils.  A  very  recent  result  shows  that  the 
two  nostrils  actually  provide  different  sensations  from  complex  mixtures 
of  odors.  Earlier  studies  that  use  simple  odors  had  failed  to  notice  this 
effect. [Lemonick] 

VIII. E.1.  Types  of  Delivery  Systems 

However,  there  are  other  kinds  of  delivery  system  that  would  be  needed 
for  other  applications  which  are  worth  pointing  out  here.  In  addition, 
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meeting  their  requirements  would  improve  the  performance  of  the 
simulation  system. 

VIII.E.I.a.  Psychophysics  System 

The  current  system  is  intended  to  provide  odors  during  simulations.  It  is 
not  intended  to  measure  the  participant’s  sense  of  smell.  It  makes  benign 
assumptions  about  how  many  levels  of  intensity  people  can  discriminate 
and  about  their  sensitivity  in  general.  However,  we  know  that  people  vary 
enormously  in  their  ability  to  detect  odors.  It  would  be  useful  to  have  a 
device  that  could  present  any  odor  in  precise  amounts  from  below  the 
threshold  than  can  be  detected  by  the  most  sensitive  person  to  above  the 
threshold  of  maximum  perceived  intensity  by  the  least  sensitive  person.  In 
between  these  minimum  and  maximum  concentrations,  it  should  be  able  to 
vary  the  concentration  more  finely  than  any  individual’s  ability  to 
discriminate  in  their  personal  range  of  sensitivity.  This  device  could  be 
used  for  screening  the  population  for  various  kinds  of  odor  blindness,  for 
testing  people’s  ability  to  improve  their  sense  of  smell,  and  for  testing 
people’s  awareness  of  odor  textures  as  opposed  to  static  odor 
concentrations. 

proprietary 
VIII.E.I.b.  Odor  Organ 

Currently,  odorants  are  mixed  as  liquids  which  is  cumbersome  and  makes 
it  difficult  for  the  perfumer  to  back  up.  There  would  be  an  important  place 
for  a  device  that  would  contain  a  large  number  of  odorous  compounds  on 
line,  be  able  to  vaporize  precise  amounts  of  those  materials,  and  to  mix 
the  vapors  in  carefully  controlled  proportions.  It  would  further  be 
desirable  to  be  able  to  vary  the  settings  very  rapidly  to  allow  the 
perfumer  to  consider  many  more  combinations  than  she  typically  can  using 
traditional  techniques. 

End  PROPRIETARY 

A  related  device  would  have  a  more  limited  but  still  appealing  application. 
Deaf-blind  individuals  are  missing  the  two  most  important  sources  of 
experience  and  might  be  very  interested  in  a  technology  that  addressed 
their  sense  of  smell.  An  odor  organ  played  with  a  keyboard  might  be  a 
source  of  pleasure  for  these  individuals.  Given  their  lack  of  other 
stimulation,  they  might  devote  themselves  to  creating  a  new  art  form 
based  on  it.  In  the  past,  the  idea  of  a  temporal  art  form  based  on  odors  has 
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always  failed  because  there  was  no  way  to  evacuate  odors  once  they  had 
been  introduced.  If  individual  delivery  devices  were  placed  in  the  seats  in 
a  theater  as  they  have  been  for  one  simulation  ride  in  Puroland  (a  theme 
park  outside  Tokyo)  and  evacuated  through  a  ceiling  exhaust  system,  it 
would  be  possible  for  compositions  created  by  deaf-blind  individuals  to  be 
performed  for  large  audiences. 

VIII.E.1.C.  Human  Interface  Device 

The  current  system  is  a  multipurpose  device  which  can  be  used  in  several 
different  virtual  reality  formats.  It  contains  attributes  such  as 
portability  and  wireless  operation  which  are  not  necessary  in  static 
applications.  In  addition,  it  provides  an  odor  repertoire  that  may  be  overly 
large  for  some  applications.  In  particular,  there  is  evidence  that  certain 
odors  may  improve  performance  in  tasks  unrelated  to  simulation.  For 
instance,  odors  are  believed  to  improve  memory,  mood,  vigilance,  spatial 
reasoning,  etc.  These  uses  of  odor  in  the  traditional  computer  interface 
and  indeed  in  any  environment  where  operator  performance  is  important 
should  be  investigated. 

In  many  cases,  a  single  odor  such  as  peppermint  is  believed  to  provide  a 
benefit.  While  it  seems  unlikely  that  peppermint  would  make  the  world  a 
better  place  indefinitely,  it  seems  that  it  might  be  a  useful  performance 
enhancer  when  used  in  particularly  important  situations  for  short  periods. 
It  is  also  possible  that  the  duration  of  the  effect  might  be  increased  by 
introducing  it  sporadically  or  according  to  a  complex  reinforcement 
schedule.  It  is  also  possible  that  a  number  of  odors  could  be  introduced  in 
patterns  that  would  vary  over  time  just  as  Muzak  is  played  in  department 
stores  to  keep  the  staff  motivated. 

A  device  targeting  this  application  might  deliver  a  single  odor  under 
computer  control.  Or  it  might  deliver  preprogrammed  sequences  of  odors. 
In  this  case,  a  single  tube  could  contain  a  series  of  different  odorants 
separated  by  bubbles  that  would  be  fed  to  a  single  transducer. 

As  more  complex  control  is  desired  and  as  the  odors  are  tied  to  particular 
tasks  and  scenarios,  a  stripped  down  version  of  the  current  device  would 
be  desired.  For  instance,  during  training  an  operator  might  learn  to 
associate  particular  odors  with  particular  tasks.  Possibly  if  the 
appropriate  odor  is  presented  when  a  particular  task  is  to  be  performed,  it 
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will  activate  the  memory  of  the  training  more  effectively.  Similarly,  if  a 
person  is  presented  with  a  complex  interface  with  many  tasks  to  perform 
and  many  contingencies  to  consider,  using  odors  to  enhance  the  user’s 
sense  of  context  might  help  him  organize  his  thinking. 

For  instance,  today’s  desktop  metaphor  claims  to  be  spatially  organized, 
but  it  is  a  fraud  unless  you  would  be  content  with  a  desk  the  size  of  an 
airline  seatback  tray.  A  real  desk  is  much  larger.  You  do  not  just  move 
your  eyes  around  it,  you  also  move  your  hands,  you  turn  your  head,  you 
refocus  your  eyes.  All  of  this  participation  strengthens  the  sense  of 
context  that  you  can  create  and  exploit  on  your  desk.  Instead  of 
overlapping  windows  which  simulate  a  pile  of  papers  not  a  desktop,  you 
could  turn  your  head  to  activate  the  window  or  tool  bar  required  for  a 
certain  task.  You  could  enhance  that  proprioceptive  cue  with  an  odor  that 
was  unique  to  the  window.  Alternatively,  odors  could  be  used  as  signals  or 
alarms  that  would  alert  the  user  to  the  need  to  perform  a  certain  set  of 
tasks  on  a  specific  set  of  inputs. 

VIII.E.I.d.  Odor  Entertainment  Systems 

Low  cost  systems  are  need  for  arcades,  movies,  home  entertainment,  cars, 
personal  space  fragrancing,  and  body  adornment.  Such  systems  will  need 
to  be  easily  reloadable  and  ideally  disposable. 

VIII. E, 2  Future  Delivery  Technology 

In  many  ways  odor  delivery  would  seem  to  be  a  perfect  problem  for  MEMS 
technology.  This  was  the  reaction  of  Ken  Gabriel  who  headed  the  MEMS 
effort  at  DARPA  when  he  was  told  of  the  Pi’s  interest  in  such  a  device  by 
Anne  Marie  Alanzillotto  of  Sarnoff  Labs,  {after  he  returned  to  CMU)  At  the 
time,  the  project  was  too  far  along  and  a  MEMS  effort  would  have  required 
at  least  an  additional  year  to  produce  a  chip.  In  addition,  the  fact  that  the 
medical  odors  were  not  available  and  their  physical  properties  therefore 
unknown  meant  that  the  requirements  could  not  be  stated  with  certainty. 
However,  the  project  was  of  sufficient  interest  that  Sarnoff  was 
interested  in  sharing  half  the  cost  of  the  effort. 

However,  there  are  many  ways  that  technology  can  be  brought  to  bear  on 
the  odor  delivery  problem  short  of  creating  the  complete  system.  First, 
low  power  methods  for  pumping  and  selecting  small  volumes  of  liquid 
odorants  are  needed.  Separately  constructed  valves  are  obviously  too 
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expensive,  too  large,  and  too  power  hungry. 

Better  methods  for  ejecting  odorants  from  the  dispenser  are  also  needed. 
The  current  system’s  100  nanoliter  bursts  are  probably  too  large.  Even  the 
25  micron  droplets  produced  by  ink  jet  printheads  are  too  large  to  inhale 
and  a  separate  transducer  would  still  be  required  to  vaporize  them.  (Ink 
jet  print  heads  were  seriously  considered  for  this  project,  but  required 
information  from  the  printer  manufacturers  that  they  were  unwilling  to 
provide.) 

Ideally,  the  dispenser  chip  would  produce  vapor  directly  instead  of 
releasing  droplets  to  be  vaporized  by  an  external  transducer.  A  possible 
problem  is  that  a  residue  might  build  up  that  would  render  the  chip 
inoperable.  Finer  control  of  the  output  volume  would  also  be  desirable. 
Tolerance  of  a  wide  variety  of  materials  and  antifouling  mechanisms 
would  be  needed. 

Also  important  is  the  issue  of  getting  odorant  to  the  chip.  Current 
technology  seems  to  permit  only  nine  liquid  input  channels  to  be 
interfaced  to  the  chip.  This  is  at  the  edge  of  what  is  considered  a  minimal 
odor  repertoire.  There  are  many  applications  that  might  require  rhore. 

In  general,  odor  delivery  is  a  perfect  application  for  MEMS  technology  in 
that  it  involves  very  small  amounts  of  material  and  addresses  a 
completely  new  application,  one  that  will  be  created  by  the  technology 
rather  than  simply  being  met  by  it.  However,  it  should  be  recognized  that 
since  better  instrumentation  is  required  in  order  to  understand  the  needs 
of  the  application,  any  advanced  delivery  system  is  going  to  require 
iterative  design  and  testing  rather  than  detailed  specification  followed  by 
a  do-it-right-the-first-time  implementation. 

Vili.F.  Telesmell 

This  application  is  quite  speculative  but  should  at  least  be  demonstrated 
to  the  level  described  earlier.  For  research  purposes,  remote  surgery  like 
simulation  should  assume  that  odors  are  necessary  until  proven  otherwise. 
Until  the  odors  needed  to  simulate  surgery  itself  are  available  and  until 
sensors  can  detect  them  as  well  as  the  human  nose,  telesmell  for  that 
application  cannot  be  meaningfully  demonstrated.  On  the  other  hand,  there 
are  sensors  that  detect  some  odors  well  enough  for  a  proof  of  concept  to 
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be  performed  for  telesmell  in  general. 

In  addition,  olfactory  telepresence  could  be  valuable  in  other  contexts.  The 
most  ubiquitous  is  telecommunication.  The  fantasy  of  teleconferencing 
systems  is  to  be  able  to  provide  a  sense  of  being  with  a  remote  person 
that  is  so  strong  that  there  is  no  desire  to  visit  him  in  order  to 
communicate  better.  One  step  was  the  concept  of  a  shared  virtual 
telecommunication  space  which  both  participants  enter  for  the  purpose  of 
the  conversation.  This  idea  which  the  PI  invented  is  now  considered  one  of 
the  fundamental  axioms  of  virtual  reality.  While  remote  individuals  are 
able  to  share  information  in  virtual  reality,  no  one  would  claim  that  they 
really  feel  like  they  are  together.  Persuasion  or  trust  building  will  require 
a  much  more  powerful  sense  of  presence. 

Possibly,  being  able  to  smell  the  other  person  would  give  you  a  better 
sense  that  you  were  with  them.  It  is  known  that  babies  react 
physiologically  to  their  mother’s  scent.  Maybe  adults  have  similar 
reactions  to  each  other’s  body  odor.  Some  people  keep  clothing  and  other 
articles  that  contain  a  loved  one’s  scent,  especially  when  they  are 
deceased.  This  idea  could  be  easily  tested  by  putting  people  in 
teleconferencing  systems  that  were  close  enough  together  that  the  odors 
from  each  space  could  be  circulated  to  the  other.  The  test  should  be  done 
with  both  intimates  and  strangers.  It  is  possible  that  the  idea  should  also 
be  tested  for  phone  conversations  between  phone  booths.  If  a  measurable 
effect  is  found,  it  might  improve  both  informal  and  personal 
telecommunication.  In  cases  where  two  people  contact  each  other  often, 
their  scents  could  be  concentrated  and  simply  transported  for  display 
during  their  conversations.  For  instance,  soldiers  deployed  in  remote 
locations  might  have  an  enhanced  sense  of  connection  with  their  families 
if  personal  odors  were  displayed  during  their  televisits. 

Body  odor  is  based  on  complex  chemicals,  but  it  is  the  opinion  of  George 
Preti,  a  chemist  at  the  Monell  Chemical  Senses  Center  that  there  are  about 
20  compounds  which  when  mixed  together  in  the  proper  proportions  define 
a  person’s  signature  scent.[Preti]  This  hypothesis  should  be  tested.  If  it  is 
true,  then  an  artificial  nose  coupled  with  a  remote  odor  delivery  system 
would  be  sufficient  to  connect  any  two  people  together  with  olfactory 
telepresence. 
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VIII.G.  Tracking  &  HMD  Display 

The  project  considered  three  optical  techniques  for  wireless  tracking  of 
HMD  participants.  It  also  briefly  considered  some  HMD  design  approaches 
that  it  had  examined  in  more  detail  in  the  time  between  the  notification 
of  the  award  and  when  the  funding  was  received.  These  efforts  were 
performed  in  collaboration  with  Dr  Jannick  Holland  of  CREOL,  the  Center 
for  Research  and  Education  in  Optics  and  Lasers,  at  the  University  of 
Central  Florida.  The  PI  was  struck  by  the  degree  to  which  optical  design  is 
still  based  on  the  techniques  of  a  hundred  years  ago  and  the  extent  to 
which  prototyping  with  these  techniques  is  prohibitively  expensive.  As  a 
result,  HMD  design  has  not  progressed  markedly  and  will  not  with 
traditional  methods.  The  fact  that  SONY'S  recent  contribution  is  based  on 
the  same  techniques  is  further  evidence  of  this  fact. 

Dr.  Holland  and  the  PI  considered  a  number  of  out-of-the-box  ideas  that 
would  accomplish  optical  functions  by  nontraditional  means.  But  real 
breakthroughs  are  needed  in  this  area  and  such  an  effort  is  appropriate  for 
DARPA  to  fund.  In  fact,  it  is  possible  that  DARPA  has  already  funded 
projects  and  caused  technology  that  has  not  yet  been  brought  to  bear.  If 
not,  such  an  effort  is  due. 

One  difficulty  was  the  interface  of  an  asymmetric  wide  field  of  view  to  a 
large  high-resolution  sensor.  The  PI  has  recently  become  aware  of  two 
related  techniques  which  offer  alternative  approaches  to  this  problem. 
The  first  is  the  use  of  switchable  LCDs  with  different  refractive  indices 
to  steer  a  laser  beam  being  considered  for  funding  by  the  Army.  The  second 
is  electronically  switchable  diffractive  optics  being  commercialized  by 
DigiLens  which  has  recently  received  $12M  in  investment.  The 
implications  of  this  technology  are  profound  to  a  wide  range  of  everyday 
as  well  as  sophisticated  problems.  DARPA  is  encouraged  to  seriously 
consider  spurring  uses  of  this  technology  and  to  take  a  bow  if  it  had  a 
hand  in  causing  it  to  be  created.  The  PI  has  several  projects  in  mind  that 
can  be  accomplished  with  this  technology  and  is  sure  that  there  are  many 
military  projects  that  would  benefit  from  it. 

Vili.H  Recommendation  Summary 

Significant  efforts  in  all  of  the  following  areas  have  been  recommended. 
These  efforts  should  not  be  pure  science  for  science  sake.  Such  purely 
scientific  work  has  led  to  a  pattern  of  omissions  that  will  persist  unless 
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practical  priorities  are  taken  into  account.  Instead,  there  should  be  a 
systematic  effort  to  look  the  olfactory  sense  from  a  applied  point  of  view 
even  when  fundamental  scientific  work  is  required. 

Human  Olfactory  Science 
Animal  Olfactory  Science 

Medical  diagnosis  by  animals  and  instruments 
Olfactory  Super  Sense 
Odor  Analysis 

odors  for  simulation 
odor  environments 
human  body  odors 
Odor  Delivery 
Odor  Simulation 

Odors  at  the  Human  Interface  Research 
Olfactory  Telepresence 
Optics  Techniques 
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iX.  Conclusion 

The  overall  conclusion  to  be  drawn  from  this  project  is  that  it  is  easy  to 
do  olfactory  display  badly  and  hard  to  do  it  a  little  bit  better.  Advances 
are  required  in  the  practical  psychophysics  of  human  perception,  the 
chemical  analysis  of  odors,  and  the  synthesis  of  odors  before  it  is 
possible  to  determine  whether  current  micro  mechanical  technology  can 
be  used  to  create  the  ultimate  odor  simulation.  The  experience  with  the 
current  system  is  that  the  best  odors  definitely  enhance  ambulatory 
simulation  now  but  that  odors  are  less  convincing  in  screen-based 
simulators. 

A  significant  effort  is  required  in  each  of  these  areas  to  bring  this 
technology  to  the  level  where  it  should  be,  given  the  state  of  our  other 
technologies.  However,  in  light  of  our  societal  prejudice  that  odors  are  not 
important  and  that  making  use  of  them  was  not  as  easy  as  we  had  hoped,  it 
is  not  unlikely  that  they  will  continue  to  be  neglected.  At  the  same  time, 
given  the  fact  that  what  is  missing  is  knowledge  of  fundamental  human 
machinery  and  that  many  applications  can  be  contemplated,  the  large 
effort  required  might  lead  to  a  large  payoff  if  it  was  undertaken. 
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